[Bitpim-devel] Timeouts on SCP-5500

[Stephen Wood]

I am experimenting with the Sanyo SCP-5500, starting with understanding
the phonebook protocol first. There are strong similarities with the
protocol for the other Sanyo phones. The headers are larger and the
packets are bigger, but the hex codes in the headers for different
packets are usually the same and the payload formats are similar. I'll
probably be able to reuse a lot of code.

When reading the phonebook, I will get a timeout at a random place. The
retry hack used for writing to the other Sanyo phones does not help. I
worry that this is a similar problem to those experienced with the
VX4400 when using the straight through cable.

Is there a clean way to trap communications timeouts so that I can try
reinitializing communications or such?

Stephen


9:41:40.729 /dev/input/ttyACM0: Timed out - flushing and trying again
9:41:43.731 /dev/input/ttyACM0: Timed out - flushing and trying again
9:41:46.733 /dev/input/ttyACM0: Timed out waiting for 7e, requested
bytes 1 - 0 bytes read
9:41:46.733 Error: Device needs attention - SCP-5500 on
/dev/input/ttyACM0
SCP-5500 on /dev/input/ttyACM0: The phone is not responding while
manipulating the phonebook.

[Roger Binns]

Post by Stephen Wood
Is there a clean way to trap communications timeouts so that I can try
reinitializing communications or such?

The problem is what to do. In the case of the VX4400, the communications
timeout happens while reading the response, and to me seems to be an
issue with buffer management on the phone. This means that it has
already done the action sent in the command, and it would be
wrong to retry some of the commands (for example the one that goes to
the next phonebook entry, deleting a file, listing the next directory etc).

The LGE driver on Windows works around the problem by sending a read URB
(USB request block) before sending any write which means the phone already
has a read buffer allocated or something.

I tried doing something similar with libusb using multi-threading and it
didn't work. I don't think libusb is designed to be multi-threaded (well
maybe version 1.0 will do the trick).

So that kind of thing can only be in a real OS level device driver.

I am out of ideas on what do about this problem. You may want to try
the device on Windows and see if the same problems occur. If they don't
then someting will be going on in the device driver to work around it.

Roger

[Stephen Wood]

Post by Roger Binns
...
The problem is what to do. In the case of the VX4400, the communications
timeout happens while reading the response, and to me seems to be an
issue with buffer management on the phone. This means that it has
already done the action sent in the command, and it would be
wrong to retry some of the commands (for example the one that goes to
the next phonebook entry, deleting a file, listing the next directory etc).

Redoing the command is not a problem for the Sanyo 5500 phonebook or
calendar. There is no state in the reading or writing of these.

The filesystem seems to work reliably.

Of course it might be a problem for media transfers since the ringers
and pictures are not in the file system.

Post by Roger Binns
I am out of ideas on what do about this problem. You may want to try
the device on Windows and see if the same problems occur. If they don't
then someting will be going on in the device driver to work around it.

My phonebook test code worked fine on Windows, using a Futuredial
driver. (I think the same driver as used for the 4900 and other sanyo
phones.)

If I capture the data with SnoopyPro, the packet I send to the phone is
a single 1034 BULK_OR_INTERRUPT_TRANSFER thingy, but the response comes
back in 4 or 5 groups. The first set of bytes almost always 99 bytes
which might be followed by 396, 198 and 341 bytes. These counts will
vary.

Stephen

Post by Roger Binns
Roger
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
Bitpim-devel mailing list
https://lists.sourceforge.net/lists/listinfo/bitpim-devel

[Roger Binns]

Post by Stephen Wood
Redoing the command is not a problem for the Sanyo 5500 phonebook or
calendar. There is no state in the reading or writing of these.

What I recommend you do is the quickest dirtiest hack to see if it
fixes the problem. Don't commit it though :-) If it does fix it
then we can try to figure out a proper way of doing this.

The reason I recommend this approach is because I tried to solve it
"properly" for the VX4400 and the low level code looked real ugly
and was an almost incomprehensible mess. And of course I discovered
that it didn't fix it anyway (since you could hit exactly the same
problem on the retry or the next command sent).

Post by Stephen Wood
The filesystem seems to work reliably.

A good test is to do an entire filesystem backup. For the 4400 I
had to add a delay of 0.3 of a second before getting each file.
If you don't the processor seems to get overwhelmed or something.

Post by Stephen Wood
If I capture the data with SnoopyPro, the packet I send to the phone is
a single 1034 BULK_OR_INTERRUPT_TRANSFER thingy, but the response comes
back in 4 or 5 groups. The first set of bytes almost always 99 bytes
which might be followed by 396, 198 and 341 bytes. These counts will
vary.

What is the bulk limit? For the LG phones and the AudioVox CDM 8900
it is 64 bytes per URB. Have you tried accessing the phone using libusb
rather than the acm module?

Roger

[Stephen Wood]

Post by Roger Binns

Post by Stephen Wood
Redoing the command is not a problem for the Sanyo 5500 phonebook or
calendar. There is no state in the reading or writing of these.

What I recommend you do is the quickest dirtiest hack to see if it
fixes the problem. Don't commit it though :-) If it does fix it
then we can try to figure out a proper way of doing this.

I'll do something. If I can keep the hack limited to com_sanyo5500, I
may check it in.

Post by Roger Binns
A good test is to do an entire filesystem backup. For the 4400 I
had to add a delay of 0.3 of a second before getting each file.
If you don't the processor seems to get overwhelmed or something.

The file system backup works fine, even without the 0.3 second delay.
But the packets in file system mode are a bit over 256 bytes, while the
phonebook protocol packets are over 1000 bytes long. (And most of that
space is not used).

Post by Roger Binns

Post by Stephen Wood
If I capture the data with SnoopyPro, the packet I send to the phone is
a single 1034 BULK_OR_INTERRUPT_TRANSFER thingy, but the response comes
back in 4 or 5 groups. The first set of bytes almost always 99 bytes
which might be followed by 396, 198 and 341 bytes. These counts will
vary.

What is the bulk limit? For the LG phones and the AudioVox CDM 8900
it is 64 bytes per URB. Have you tried accessing the phone using libusb
rather than the acm module?

How do I get the bulk limit? When I was trying the Sanyo 4900 using
libusb, things would seem to happen in 64 byte chunks. It also always
eventually timed out on me.

When I try the libusb with the 5500, it will sometime work, but often
get a usbdevicewrapper instance has no attribute 'flushInput' error.

Traceback (most recent call last):
File "./gui.py", line 150, in run
res=call()
File "./gui.py", line 91, in __call__
return apply(self.method, self.args+args, d)
File "./gui.py", line 1067, in getdata
i[1](results)
File "./com_sanyo5500.py", line 69, in getphonebook
self.sendpbcommand(req, self.protocolclass.studyresponse,
writemode=True)
File "./com_sanyo.py", line 186, in sendpbcommand
data=self.comm.readuntil(self.pbterminator, logsuccess=False,
numfailures=numretry)
File "./commport.py", line 267, in readuntil
self.ser.flushInput()
AttributeError: _usbdevicewrapper instance has no attribute 'flushInput'


Stephen

[Roger Binns]

Post by Stephen Wood
I'll do something. If I can keep the hack limited to com_sanyo5500, I
may check it in.

Ideally I would like a generic solution that can work for all phones.
So far I don't have a solution that works for ANY phone!

Post by Stephen Wood
How do I get the bulk limit? When I was trying the Sanyo 4900 using
libusb, things would seem to happen in 64 byte chunks. It also always
eventually timed out on me.

It is wMaxPacketSize in an endpoint descriptor. This is from an AudioVox
8900.

Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type none
wMaxPacketSize 64
bInterval 0

Post by Stephen Wood
File "./commport.py", line 267, in readuntil
self.ser.flushInput()
AttributeError: _usbdevicewrapper instance has no attribute 'flushInput'

Mmmm, that code is somewhat dodgy anyway since it flushes the input
buffer, which would throw away any data in it. However we just
timed out on the read meaning there is no data to flush. The win32
pySerial code does a fair job of resetting/kicking the input.

I added a flushInput method to the usbdevicewrapper. It has a call to
native.usb.USBFile.resetep. You can see commented out code in that
that actually clears halts and does and endpoint reset at the USB
level. That had no effect on the VX4400 USB issues as there wasn't
actually a halt, nor did resetep actually have any effect. You can
try uncommenting all that to see if it makes a difference.

Roger

[Stephen Wood]

Post by Roger Binns

Post by Stephen Wood
I'll do something. If I can keep the hack limited to com_sanyo5500, I
may check it in.

Ideally I would like a generic solution that can work for all phones.
So far I don't have a solution that works for ANY phone!

I made a version of sendpbcommand in com_sanyo5500 that seems to get
around timeouts. I simply rewrite the output packet and wait for the
reply again. This can work for the sanyo phones since there are no init
and nextentry as there are for the LG phones.

Post by Roger Binns

Post by Stephen Wood
How do I get the bulk limit? When I was trying the Sanyo 4900 using
libusb, things would seem to happen in 64 byte chunks. It also always
eventually timed out on me.

It is wMaxPacketSize in an endpoint descriptor. This is from an AudioVox
8900.
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type none
wMaxPacketSize 64
bInterval 0

T: Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 9 Spd=12 MxCh= 0
D: Ver= 1.01 Cls=02(comm.) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
P: Vendor=0474 ProdID=0701 Rev= 0.00
S: Manufacturer=SANYO Electric Co. Ltd.
S: Product=SANYO USB Phone
S: SerialNumber=Serial Number
C:* #Ifs= 2 Cfg#= 1 Atr=e0 MxPwr=100mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=02 Prot=01 Driver=acm
E: Ad=81(I) Atr=03(Int.) MxPS= 16 Ivl=32ms
I: If#= 1 Alt= 0 #EPs= 2 Cls=0a(data ) Sub=00 Prot=00 Driver=acm
E: Ad=8a(I) Atr=02(Bulk) MxPS= 64 Ivl=0ms
E: Ad=0b(O) Atr=02(Bulk) MxPS= 64 Ivl=0ms

I think the Sanyo 5500 is also 64 characters.

Stephen

Post by Roger Binns
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
Bitpim-devel mailing list
https://lists.sourceforge.net/lists/listinfo/bitpim-devel

[Roger Binns]

Post by Stephen Wood
I made a version of sendpbcommand in com_sanyo5500 that seems to get
around timeouts. I simply rewrite the output packet and wait for the
reply again. This can work for the sanyo phones since there are no init
and nextentry as there are for the LG phones.

When you are happy with it, can you put it back in the main code for all
phones :-) Maybe add a safe to retry parameter to the routine which would
always be yes for Sanyo, and no for Brew and LG stuff?

Post by Stephen Wood
E: Ad=8a(I) Atr=02(Bulk) MxPS= 64 Ivl=0ms
E: Ad=0b(O) Atr=02(Bulk) MxPS= 64 Ivl=0ms
I think the Sanyo 5500 is also 64 characters.

That is what it looks like for those endpoints. It doesn't quite explain
why Snoopy was claiming packets larger than that since that would be a
violation of the USB protocol.bp

The USBFile code in BitPim chops up USB requests and responses to be
the max size (or the size requested if smaller). Maybe the kernel
internals do the same as well.

Roger

[Stephen Wood]

Post by Roger Binns

Post by Stephen Wood
I made a version of sendpbcommand in com_sanyo5500 that seems to get
around timeouts. I simply rewrite the output packet and wait for the
reply again. This can work for the sanyo phones since there are no init
and nextentry as there are for the LG phones.

When you are happy with it, can you put it back in the main code for all
phones :-) Maybe add a safe to retry parameter to the routine which would
always be yes for Sanyo, and no for Brew and LG stuff?

Are you proposing to merge sendbrewcommand, the LG sendpbcommand and the
Sanyo sendpbcommand into one routine? They are all similar, so I guess
that's possible. The Brew error reporting in sendbrewcommand should be
safe for Phonebook calls since presumably there are no phonebook
protocol packets starting with 0x59. Presumably I can also figure out
how to bypass the sequence stuff in the LG sendpbcommand by testing to
see if request.header.sequence exists. At any rate, there is stuff in
the LG sendcommand, like CRC checking, that you added since I forked the
Sanyo one that I should have in the Sanyo code.

Post by Roger Binns

Post by Stephen Wood
E: Ad=8a(I) Atr=02(Bulk) MxPS= 64 Ivl=0ms
E: Ad=0b(O) Atr=02(Bulk) MxPS= 64 Ivl=0ms
I think the Sanyo 5500 is also 64 characters.

That is what it looks like for those endpoints. It doesn't quite explain
why Snoopy was claiming packets larger than that since that would be a
violation of the USB protocol.

Is there anyther usb monitor program that you recommend other than
snoopypro?

Stephen

[Roger Binns]

Post by Stephen Wood
Are you proposing to merge sendbrewcommand, the LG sendpbcommand and the

The LG commands are quite different from the brew ones, with different
numbers of bytes of prefix, different values for those bytes, a
different means of error reporting etc. The only thing they have in
common with the brew ones is the crc and ~ terminator. That is why
the LG and brew ones are similar but very different.

Post by Stephen Wood
At any rate, there is stuff in
the LG sendcommand, like CRC checking, that you added since I forked the
Sanyo one that I should have in the Sanyo code.

Maybe the routine can be split in two with the framing, crc checking etc handled
as common code and a higher level that does the actual composition of
bytes, generates exceptions on return etc?

Post by Stephen Wood
Is there anyther usb monitor program that you recommend other than
snoopypro?

I think I used ordinary snoopy, but was very unimpressed.

Roger