Discussion:
[Bitpim-devel] Error processing & operator feedback
d***@netzero.com
2004-10-29 21:44:38 UTC
Permalink
Quick question: do we have any convention and/or guidlines wrt error processing of the info being transferred to the phone. A good example would be what Vic had previously pointed out for Samsung phones: a phonebook entry name that's too long. What's BP supposed to do in that case: truncate the name and send it out, or not sending the entry and somehow notify the user? Also, do we have something in place for savephonebook, savecalendar, etc routine to display the results and/or errors to users (I'm thinking of maybe a dict entry in the result param passed back to gui for it to do the displaying of the info).

Or is it up to individual coders to handle it?

Thanks,

-Joe Pham


________________________________________________________________
Speed up your surfing with NetZero HiSpeed.
Now includes pop-up blocker!
Only $14.95/ month - visit http://www.netzero.com/surf to sign up today!
Stephen Wood
2004-10-30 03:36:55 UTC
Permalink
I am not sure if I do it completely correctly in the Sanyo code, but the
save routines are supposed to return a copy of the phonebook as it ends
up on the phone. (As if doing a overwrite read after writing to the
phone.) This can remove/change information from what was being held by
BitPim, but the result is that what is in BitPim reflects what is in the
phone.

For the Sanyo code, I think I do the phonebook correctly, but for the
calendar and media, I don't modify BitPims information.
Post by d***@netzero.com
Quick question: do we have any convention and/or guidlines wrt error processing of the info being transferred to the phone. A good example would be what Vic had previously pointed out for Samsung phones: a phonebook entry name that's too long. What's BP supposed to do in that case: truncate the name and send it out, or not sending the entry and somehow notify the user? Also, do we have something in place for savephonebook, savecalendar, etc routine to display the results and/or errors to users (I'm thinking of maybe a dict entry in the result param passed back to gui for it to do the displaying of the info).
Or is it up to individual coders to handle it?
Thanks,
-Joe Pham
Roger Binns
2004-10-30 21:10:26 UTC
Permalink
Post by Stephen Wood
I am not sure if I do it completely correctly in the Sanyo code, but
the save routines are supposed to return a copy of the phonebook as
it ends up on the phone. (As if doing a overwrite read after writing
to the phone.) This can remove/change information from what was
being held by BitPim, but the result is that what is in BitPim
reflects what is in the phone.
That doesn't actually happen any more :-) The only information looked
at are serial updates (since you don't know what serial will be assigned
or changed until a record is written to the phone).

However it will need to happen again at some point in the future
in order to generate deltas for doing syncing. For example, if
BitPim has the name for a record as "John ReallyReallyLongName"
and we later read "John ReallyReal" then we need to be able to
distinguish from phone limitations vs the user explicitly editing
the name on the phone.

Note that it isn't only a matter of truncation. The name could
also have character changes (eg there could be accented characters
stored in the BitPim record that are stripped when sent to the
phone). This kind of thing could even happen with phone numbers.
eg BitPim could be storing it as "1234567890 x 1234" and the phone
only has "1234567890". It could even be the other way round with
the phone storing extra characters (PT*).

When we move to SQLite, a snapshot of information will be taken as
it was written to the phone so we can later make the deltas.

Roger
Vic Heintz
2004-10-30 12:13:57 UTC
Permalink
Post by Stephen Wood
I am not sure if I do it completely correctly in the Sanyo code, but
the
save routines are supposed to return a copy of the phonebook as it ends
up on the phone. (As if doing a overwrite read after writing to the
phone.) This can remove/change information from what was being held by
BitPim, but the result is that what is in BitPim reflects what is in
the
phone.
If you are saying that BitPim's data should always reflect what is in
the phone, what happens if someone has two different phones and they
switch their phone selection? The requirements change then. Will BitPim
be aware that the data it has is for a different phone than the one
currently selected? Do the import procedures need to be aware of the
phone model to keep fields within their bounds for that phone? Should
the gui restrict what you type so you don't for example type more
characters for a name than is allowed by that phone?

Vic
Roger Binns
2004-10-30 21:11:13 UTC
Permalink
Post by Vic Heintz
If you are saying that BitPim's data should always reflect what is in
the phone, what happens if someone has two different phones and they
switch their phone selection?
See the message I just sent. BitPim copes fine with multiple phones.

Roger
Vic Heintz
2004-10-30 15:45:28 UTC
Permalink
I know that Stephen has done some work using the Wallpaper feature of
BitPim to download camera images from a Samsung A620. I have tested his
code and been similarly successful with my A670. This brings up a
general issue that perhaps needs to be addressed. Is there a plan for
BitPim to handle two distinct types of images: true wallpapers and
camera phone photos? Are any of the currently supported phones camera
phones?

On the Samsung A670 (and the A610) the camera pictures taken are stored
in the filesystem in the directory
/digital_cam. They are up to 640 X 480 jpg files named Image001, etc
with no extension and are stored internally with an extra header. They
can be downloaded in BitPim but there probably should never be any
attempts to upload them to the phone. They can be selected from the
"Gallery" menu on the phone and used as a background wallpaper or as a
"caller ID" image.

On the other hand wallpaper pictures are 128 X 128 pixel files with a
bmp extension and are stored in the directory /brew/shared. They retain
the file name they had when downloaded from Verizon or uploaded over
the data cable. They can be selected from the "My Image" menu for use
as a background wallpaper image but do not seem to be available as
Picture ID images. (There is a "My Image" menu for this but these
images must live in a different directory from /brew/shared.)

I see that the BitPim Wallpaper tab has a property called "Origin". Can
this be or is this being used to distinguish between the different
types of images? If so then maybe I am covering ground that has already
been trodden.

Vic
d***@netzero.com
2004-10-30 16:16:38 UTC
Permalink
With this approach, BP would wipe out any invalid entries that users inadvertently made. Users can always re-enter the info, but probably needs to be told why. Still sounds kinda harsh. How 'bout some indication on the display of those invalid entries that have not been successfully sent to the phone (red font for example) and give users a chance to correct them.

Just a thought.

-Joe Pham

-- Stephen Wood <***@bitpim.org> wrote:
I am not sure if I do it completely correctly in the Sanyo code, but the
save routines are supposed to return a copy of the phonebook as it ends
up on the phone. (As if doing a overwrite read after writing to the
phone.) This can remove/change information from what was being held by
BitPim, but the result is that what is in BitPim reflects what is in the
phone.

For the Sanyo code, I think I do the phonebook correctly, but for the
calendar and media, I don't modify BitPims information.
Post by d***@netzero.com
Quick question: do we have any convention and/or guidlines wrt error processing of the info being transferred to the phone. A good example would be what Vic had previously pointed out for Samsung phones: a phonebook entry name that's too long. What's BP supposed to do in that case: truncate the name and send it out, or not sending the entry and somehow notify the user? Also, do we have something in place for savephonebook, savecalendar, etc routine to display the results and/or errors to users (I'm thinking of maybe a dict entry in the result param passed back to gui for it to do the displaying of the info).
Or is it up to individual coders to handle it?
Thanks,
-Joe Pham
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Bitpim-devel mailing list
Bitpim-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitpim-devel


________________________________________________________________
Speed up your surfing with NetZero HiSpeed.
Now includes pop-up blocker!
Only $14.95/ month - visit http://www.netzero.com/surf to sign up today!
Roger Binns
2004-10-30 21:05:17 UTC
Permalink
Post by d***@netzero.com
Quick question: do we have any convention and/or guidlines wrt error
processing of the info being transferred to the phone. A good
example would be what Vic had previously pointed out for Samsung
phones: a phonebook entry name that's too long. What's BP supposed
The phonebook writing happens in several stages:

1: GetFundamentals is called. This interacts with the phone and
get any phone specific information you need. Amongst the useful
things to get is the phone serial number.

2: Your phone profile's convertphonebooktophone method is called.
This method should take the phonebook formatted information
and convert it into whatever is needed for the phone.

3: You phone savephonebook method is called to interact with the
phone and update the content. You should also record any
serial number updates.

4: Back in BitPim, the underlying data is modified with any serial
number updates.

(Note that if you are also writing out calendar, wallpaper, ringtone
etc then steps 2, 3 and 4 will also be done for the data types. They
will all be done at once - eg step 2 for phonebook, step 2 for calendar,
step 2 for ringers, then step 3 for phonebook, step 3 for calendar etc).

Step 1 would be where you consult the phone for dynamic information
(eg if the number of phonebook entries depended on some other factor
and wasn't a constant.) It will also be when various config settings
will be established in the future (eg if you store the names in
First Last or Last First format).

Step 2 is where you would truncate fields, constrain the number of
entries etc. It should do all the data manipulation necessary so that
step 3 won't fail unless there are serious problems. In the future
user interaction will be possible here. I expect that all sorts of
data truncation will happen here, from shortening fields to ignoring
others (eg if the phone only supports one email address but the record
has 4 values). They can all be shown to the user in one go before the
user confirms sending the data to the phone. (We'd probably need some
sorting on severity since I usually won't care that "United Airlines"
has been truncated to "United Airli" for my phone).

Step 3 should be doing a minimum amount of work, basically just dumping
the data already pre-formatted by step 2 into the phone. This should be
as simple as possible, and no user interaction will be allowed (or possible).

Step 4 is when BitPim updates it's own internal data.

I don't think the Samsung or Sanyo phones have serial numbers, so I'll
outline them now. Well designed data sources assign a unique id to
each record. If the record is deleted and another created then it gets
a new unique id. Sync programs can then use the unique id to match records
and changes to them. For example if serial 12 was "John Smith" last week
and now has a name of "John Smythe" and a different phone number now, you
know it is still the same record, and the user has updated their data.
Without a serial you are just guessing.

The LG phones use serial numbers (a 32 bit quantity) so you can do this
syncing easily. Outlook, Evolution, Apple Addressbook etc all use them
as well, except those programs typically use a larger sized value.
BitPim also uses one.

Every phonebook record has a list of serials associated with it, for the
various data sources that the record has come from. If the record also came
from a phone, then there will be one for that including the phone's unique id.
This makes it considerably easier to match the same record when importing
and exporting to various data sources (which includes the phones).

The serial information has a huge effect on the match in the import dialog,
but it isn't treated as absolute truth. It is possible for a data source
to assign numbers again. (The LG phones will do so if you reset them to a
clean state).

The slot numbers on the Audiovox and Sanyo phones, and it seems Samsung phones
could be used as a serial as well. Basically if there is a very high probability
that what was in slot 3 last week and is in slot 3 now is the same record, even
if the information is different then it is useful. It is probably worth
experimenting with.

Here is an example from some of my test data. I have it on Outlook and on my
VX4400 and VX6000 phones:

'serials':
[
{
'serial1': 193,
'serial2': 193,
'sourcetype': 'lgvx4400',
'sourceuniqueid': '1aadcbb3d92021e92b614f326e2de9f3cb1f77d2',
},
{
'id': '73c7bf16536c02928899f3dc8deb13862490a4dd',
'sourcetype': 'bitpim',
},
{
'EntryID': u'000000009FD24B3CF1F64841816E3F7CCCF153D704332000',
'FolderID': '000000009FD24B3CF1F64841816E3F7CCCF153D742810000',
'sourcetype': 'outlook',
},
{
'serial1': 487,
'serial2': 487,
'sourcetype': 'lgvx6000',
'sourceuniqueid': '4af9c590b64261f5fa408a1ea793eec0b5b9a6da',
},
],

If I had more than one VX4400, then there would be another entry for that
one, but with a different sourceuniqueid.
Post by d***@netzero.com
Or is it up to individual coders to handle it?
There is a plan, but nothing has been implemented yet. It may be worthwhile
doing a mockup screenshot first with what you would expect to show the user.
I know that as a user I wouldn't care the second time about data truncation
etc :-)

Roger
d***@netzero.com
2004-10-31 00:06:54 UTC
Permalink
Very well written explanation. I assume that this is what you think what should happen but not all implemented right now (for example: as far as I could tell, phonebook is the only one that has a convertor func for step 2). If one were to start following this approach (maybe as a test case), what would be the first thing that you think should be added? Also, what's your take on some other ideas such as flagging invalid entries and providing feedbacks to users after each phone get/send operation?

-Joe Pham
Post by d***@netzero.com
Quick question: do we have any convention and/or guidlines wrt error
processing of the info being transferred to the phone. A good
example would be what Vic had previously pointed out for Samsung
phones: a phonebook entry name that's too long. What's BP supposed
The phonebook writing happens in several stages:

1: GetFundamentals is called. This interacts with the phone and
get any phone specific information you need. Amongst the useful
things to get is the phone serial number.

2: Your phone profile's convertphonebooktophone method is called.
This method should take the phonebook formatted information
and convert it into whatever is needed for the phone.

3: You phone savephonebook method is called to interact with the
phone and update the content. You should also record any
serial number updates.

4: Back in BitPim, the underlying data is modified with any serial
number updates.

(Note that if you are also writing out calendar, wallpaper, ringtone
etc then steps 2, 3 and 4 will also be done for the data types. They
will all be done at once - eg step 2 for phonebook, step 2 for calendar,
step 2 for ringers, then step 3 for phonebook, step 3 for calendar etc).

Step 1 would be where you consult the phone for dynamic information
(eg if the number of phonebook entries depended on some other factor
and wasn't a constant.) It will also be when various config settings
will be established in the future (eg if you store the names in
First Last or Last First format).

Step 2 is where you would truncate fields, constrain the number of
entries etc. It should do all the data manipulation necessary so that
step 3 won't fail unless there are serious problems. In the future
user interaction will be possible here. I expect that all sorts of
data truncation will happen here, from shortening fields to ignoring
others (eg if the phone only supports one email address but the record
has 4 values). They can all be shown to the user in one go before the
user confirms sending the data to the phone. (We'd probably need some
sorting on severity since I usually won't care that "United Airlines"
has been truncated to "United Airli" for my phone).

Step 3 should be doing a minimum amount of work, basically just dumping
the data already pre-formatted by step 2 into the phone. This should be
as simple as possible, and no user interaction will be allowed (or possible).

Step 4 is when BitPim updates it's own internal data.

I don't think the Samsung or Sanyo phones have serial numbers, so I'll
outline them now. Well designed data sources assign a unique id to
each record. If the record is deleted and another created then it gets
a new unique id. Sync programs can then use the unique id to match records
and changes to them. For example if serial 12 was "John Smith" last week
and now has a name of "John Smythe" and a different phone number now, you
know it is still the same record, and the user has updated their data.
Without a serial you are just guessing.

The LG phones use serial numbers (a 32 bit quantity) so you can do this
syncing easily. Outlook, Evolution, Apple Addressbook etc all use them
as well, except those programs typically use a larger sized value.
BitPim also uses one.

Every phonebook record has a list of serials associated with it, for the
various data sources that the record has come from. If the record also came
from a phone, then there will be one for that including the phone's unique id.
This makes it considerably easier to match the same record when importing
and exporting to various data sources (which includes the phones).

The serial information has a huge effect on the match in the import dialog,
but it isn't treated as absolute truth. It is possible for a data source
to assign numbers again. (The LG phones will do so if you reset them to a
clean state).

The slot numbers on the Audiovox and Sanyo phones, and it seems Samsung phones
could be used as a serial as well. Basically if there is a very high probability
that what was in slot 3 last week and is in slot 3 now is the same record, even
if the information is different then it is useful. It is probably worth
experimenting with.

Here is an example from some of my test data. I have it on Outlook and on my
VX4400 and VX6000 phones:

'serials':
[
{
'serial1': 193,
'serial2': 193,
'sourcetype': 'lgvx4400',
'sourceuniqueid': '1aadcbb3d92021e92b614f326e2de9f3cb1f77d2',
},
{
'id': '73c7bf16536c02928899f3dc8deb13862490a4dd',
'sourcetype': 'bitpim',
},
{
'EntryID': u'000000009FD24B3CF1F64841816E3F7CCCF153D704332000',
'FolderID': '000000009FD24B3CF1F64841816E3F7CCCF153D742810000',
'sourcetype': 'outlook',
},
{
'serial1': 487,
'serial2': 487,
'sourcetype': 'lgvx6000',
'sourceuniqueid': '4af9c590b64261f5fa408a1ea793eec0b5b9a6da',
},
],

If I had more than one VX4400, then there would be another entry for that
one, but with a different sourceuniqueid.
Post by d***@netzero.com
Or is it up to individual coders to handle it?
There is a plan, but nothing has been implemented yet. It may be worthwhile
doing a mockup screenshot first with what you would expect to show the user.
I know that as a user I wouldn't care the second time about data truncation
etc :-)

Roger


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Bitpim-devel mailing list
Bitpim-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitpim-devel


________________________________________________________________
Speed up your surfing with NetZero HiSpeed.
Now includes pop-up blocker!
Only $14.95/ month - visit http://www.netzero.com/surf to sign up today!
Roger Binns
2004-10-31 06:04:39 UTC
Permalink
Post by d***@netzero.com
Very well written explanation. I assume that this is what you think
what should happen
That is correct.
Post by d***@netzero.com
but not all implemented right now (for example: as
far as I could tell, phonebook is the only one that has a convertor
func for step 2).
Correct.
Post by d***@netzero.com
If one were to start following this approach
(maybe as a test case), what would be the first thing that you think
should be added?
The main aim is to get the phonebook stuff working really well under
the hood, and then copying the design to the other data sources, as
well as a migration to SQLite at some point to store temporal information.

The changes would need to be as follows:

- Split out the "helper" functions from phonebook.PhoneWidget into
a seperate class/instance. This is the object passed as the
helper parameter to convertphonebooktophone.

- Add various functions to it for recording the truncation and omission
of information (this is why most of the truncation/omission is done by
by methods within helper so that they can automatically
record the data mangling)

- Add a method to phonebook.PhoneWidget that can take the log of what
mangling is happening, prioritise and present to the user giving
them a chance to cancel the write
Post by d***@netzero.com
Also, what's your take on some other ideas such as
flagging invalid entries
DataPilot does this in a very good way, which I assume they got straight
from the work of Alan Cooper. Here is a screenshot of how they show
a field with too long a value:

Loading Image...

Attached is a dialog box when the selected phone doesn't support some
of the fields. This kind of feedback is the best I have seen. However
implementing it will require the UI code being able to find out the
limitations from the low level phone code.

We would need to be able to describe this in quite a rich way. For
example the LG phones require a record have at least one phone number
or at least one email address. Probably the easiest way of doing it
would be for the Profile class to have functions that are called
with bits of data and they return how the various fields should be
shown.
Post by d***@netzero.com
and providing feedbacks to users after each phone get/send operation?
What feedback should be shown after a get? BitPim already shows you
how the information will be merged with your existing information,
and allows you to control that and edit it before accepting the results.

As for phone sending, my principle is that BitPim should confirm with
you before mangling data on the phone rather than afterwards :-)

Roger
d***@netzero.com
2004-11-01 17:40:17 UTC
Permalink
Just curious, what do you see as the advantages of splitting out the helper funcs into a separate class, other than having a cleaner design/code?

-Joe Pham
Post by d***@netzero.com
Very well written explanation. I assume that this is what you think
what should happen
That is correct.
Post by d***@netzero.com
but not all implemented right now (for example: as
far as I could tell, phonebook is the only one that has a convertor
func for step 2).
Correct.
Post by d***@netzero.com
If one were to start following this approach
(maybe as a test case), what would be the first thing that you think
should be added?
The main aim is to get the phonebook stuff working really well under
the hood, and then copying the design to the other data sources, as
well as a migration to SQLite at some point to store temporal information.

The changes would need to be as follows:

- Split out the "helper" functions from phonebook.PhoneWidget into
a seperate class/instance. This is the object passed as the
helper parameter to convertphonebooktophone.

- Add various functions to it for recording the truncation and omission
of information (this is why most of the truncation/omission is done by
by methods within helper so that they can automatically
record the data mangling)

- Add a method to phonebook.PhoneWidget that can take the log of what
mangling is happening, prioritise and present to the user giving
them a chance to cancel the write
Post by d***@netzero.com
Also, what's your take on some other ideas such as
flagging invalid entries
DataPilot does this in a very good way, which I assume they got straight
from the work of Alan Cooper. Here is a screenshot of how they show
a field with too long a value:

http://www.rogerbinns.com/vx4400/img/dplongfield.png

Attached is a dialog box when the selected phone doesn't support some
of the fields. This kind of feedback is the best I have seen. However
implementing it will require the UI code being able to find out the
limitations from the low level phone code.

We would need to be able to describe this in quite a rich way. For
example the LG phones require a record have at least one phone number
or at least one email address. Probably the easiest way of doing it
would be for the Profile class to have functions that are called
with bits of data and they return how the various fields should be
shown.
Post by d***@netzero.com
and providing feedbacks to users after each phone get/send operation?
What feedback should be shown after a get? BitPim already shows you
how the information will be merged with your existing information,
and allows you to control that and edit it before accepting the results.

As for phone sending, my principle is that BitPim should confirm with
you before mangling data on the phone rather than afterwards :-)

Roger


________________________________________________________________
Speed up your surfing with NetZero HiSpeed.
Now includes pop-up blocker!
Only $14.95/ month - visit http://www.netzero.com/surf to sign up today!
Roger Binns
2004-11-02 06:20:39 UTC
Permalink
Post by d***@netzero.com
Just curious, what do you see as the advantages of splitting out the
helper funcs into a separate class, other than having a cleaner
design/code?
At the moment the functions are all part of the actual phone ui class.

I do want BitPim to be able to work in a command line mode as well,
so that in future it could be used from other programs. That is why
the main entry point is bp.py which looks at command line options and
calls gui.py.

If you haven't already read it, now would be a good time to have a look
at examples/phonespec.html

That doc could do with some updating, and I'd be happy for any :-)

Roger
d***@netzero.com
2004-11-02 22:36:33 UTC
Permalink
That makes more sense. And yes, I did read phonespec.html and found it to be most helpful. Some example code snippets here & there would probably be more helpful.

-Joe Pham
Post by d***@netzero.com
Just curious, what do you see as the advantages of splitting out the
helper funcs into a separate class, other than having a cleaner
design/code?
At the moment the functions are all part of the actual phone ui class.

I do want BitPim to be able to work in a command line mode as well,
so that in future it could be used from other programs. That is why
the main entry point is bp.py which looks at command line options and
calls gui.py.

If you haven't already read it, now would be a good time to have a look
at examples/phonespec.html

That doc could do with some updating, and I'd be happy for any :-)

Roger


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Bitpim-devel mailing list
Bitpim-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitpim-devel


________________________________________________________________
Speed up your surfing with NetZero HiSpeed.
Now includes pop-up blocker!
Only $14.95/ month - visit http://www.netzero.com/surf to sign up today!
Roger Binns
2004-11-03 06:33:53 UTC
Permalink
Post by d***@netzero.com
That makes more sense. And yes, I did read phonespec.html and found
it to be most helpful. Some example code snippets here & there would
probably be more helpful.
I would rather point to real code than have example code that gets out
of date in phonespec.html.

Now that you understand more of how BitPim works, please send me updates
for the phonespec file - especially stuff you wish had been in there
when you read it.

Roger

Loading...