[Bitpim-devel] testdata generator added

[Roger Binns]

This will be of most use to Stephen. In the experiments directory there
is a new script gentestdata. Recommended way of running it is:

cd experiments
python gentestdata.py ../tests
cd ..

(../tests is the directory it creates the data in). There is a nice
csv file for importing into the phonebook. (Note it may take several
minutes to generate). There are also midi files and wallpapers. There
is also a calendar index.idx file you can place in your data directory.

As a result of playing with this test data, I have found numerous areas
where code needs to be tightened up (it is usually because it doesn't
truncate fields when formatting data for the phone).

You can also turn on profiling in bp.py (change the False to True).
I managed to significantly improve startup and import speeds with
large data sets by changing a few lines.

Roger

[Stephen Wood]

Post by Roger Binns
This will be of most use to Stephen. In the experiments directory there
cd experiments
python gentestdata.py ../tests
cd ..

Trying to import this test data will be most interesting! As well as
help to find outright bugs, it will force me to deal with several
issues.

1. Calendar stuff Sanyo doesn't support: daybitmaps, whatever they
are, exceptions, repeat codes (monfri).

Do I drop the entry altogether, or do my best to put it in the phone
somehow.

How do I notify the user what has been done?

2. Calendar entry limits. Sanyo supports 100 entries.

What to do if more than 100 dates in BitPim? Take first 100. (Present
behavior) Sort by date and take first 100 after current date and then
take expired entries working backward from present time?

3. Test behaviour with more than 300 names in phone book.

4. Sanyo advertises 300 names, 500 total numbers in phone. Each name
can hold 7 numbers. 300*7 = 2100 > 500. What to do if trying to write

Post by Roger Binns
500 numbers, but less than 300 names?

The 500 number limit is from the Caller ID buffer in the phone. The
present code will allow more than 500 numbers, but only put the first
500 numbers in the caller id buffer. (The numbers are sorted by type,
first all the home phone numbers are written to the caller id buffer,
then all the work, mobile etc.) But I have never actually written > 500
phone numbers to my phone. With the test data I can find out if A) It
works, B) the phonebook on the phone is at all functional afterwards.

Stephen

[Roger Binns]

Post by Stephen Wood
1. Calendar stuff Sanyo doesn't support: daybitmaps, whatever they
are, exceptions, repeat codes (monfri).
Do I drop the entry altogether, or do my best to put it in the phone
somehow.
How do I notify the user what has been done?

The first bit of good news is I documented all the fields at the top of
bpcalendar.py. The whole format needs several bits of fixing and
tweaking as noted in the comments. The code path will need to be
changed so that it is similar to the codepath for the phonebook.
That will allow the code in the profile that does the conversion
for the phone to generate warnings and errors etc.

My goal is to get the phonebook working well in 0.7 (that includes
import and export - but not sync which is a harder problem). I'll
also need to rework the wallpaper and ringtone widgets since they are
somewhat useless at the moment. The calendar will be fixed properly
for 0.8.

Post by Stephen Wood
2. Calendar entry limits. Sanyo supports 100 entries.
What to do if more than 100 dates in BitPim? Take first 100. (Present
behavior) Sort by date and take first 100 after current date and then
take expired entries working backward from present time?

The helper class for 0.8 will let you ask for the n most relevant
entries pretty much using an algorithm substantially similar to what
you just descibed.

Have you tried playing with test data so far?

Roger

[Stephen Wood]

Post by Roger Binns
Have you tried playing with test data so far?
Roger

Just tried it. The sanyo code first stumbles on a phone number longer
than the field size of 49. How do you enforce the size limits for
fields in the LG code?

Stephen

...
File "./com_sanyo.py", line 325, in makeentry
e.numbers[numberindex].number=entry[k][i]
File "./p_sanyo.py", line 6955, in __setfield_number
self.__field_number=STRING(value,**dict)
File "./prototypes.py", line 277, in __init__
self._update(args,kwargs)
File "./prototypes.py", line 306, in _update
raise ValueLengthException(l, self._sizeinbytes)
ValueLengthException: The value (length 243) is the wrong size for space
49

[Roger Binns]

Post by Stephen Wood
The sanyo code first stumbles on a phone number longer
than the field size of 49. How do you enforce the size limits for
fields in the LG code?

I truncate the number and added a comment about how I really
should decide what to do about it :-)

In reality the only numbers that are likely to be affected are
ones made using calling cards where you have the access number,
card number, pin number, and dialed number. I'll probably
have to start asking users want they want to do.

Roger