[Bitpim-devel] serial comm update

[Tim Lowery]

Roger,

Enclosed are some candidate replacement routines that pertain to serial
communications. I looked at the pyserial code, but that I didn't see
anything that would explain dropping of bytes during read operations or
the serial open port failure. Given that the phone firmware, device
drivers, and cables may be contributing to the problem, I decided to
modify some of the serial routines to perform simple retries and
checksum verification.

There changes seem to eliminate all of my serial problems such as
"could not open port" or the "8-6-2" alert. Occasionally I'll
encounter "segmentation fault" while fetching the phone data or
expanding file info. The application promptly aborts when this occurs,
and it's hard to figure out why. If you have any ideas about narrowing
this down, let me know.

I modified sendpbcommand and sendbrewcommand to perform a single retry,
but only when the baud rate is 38400 or below. At 115200 and above it
always times out, so retries are ineffective. You might wish to review
the exception handling and how it relates to the application overall.
Also, included is a change in the CommConnection init routine.
Collectively, these changes have made a significant improvement in
serial reliability.

Also, I noticed that the phone must be set to RS-232 mode when
retrieving version info. This seems to be the only case when serial
communications is dependent on the mode setting in the phone.
Otherwise, during the switch to modem mode, timeouts will occur.

The modified routines are shown below.

Tim

----------------------------------

def sendpbcommand(self, cmd, data):
# Only repeat if baudrate is low. At higher baudrates timeouts
# frequently occur, so retrying doesn't really help.
if self.comm.ser.ispeed<=38400:
numTimes=2
else:
numTimes=1

# Repeat the indicated number of times
for retry in range(numTimes):
# Construct and send commmand message
msg="\xff"+chr(cmd)+chr(self.seq&0xff)+data
msg=self.escape(msg+self.crcs(msg))+self.terminator
self.seq+=1
self.comm.write(msg)

# Retrieve response from phone
try:
res=self.unescape(self.comm.readuntil(self.terminator))
except commport.CommTimeout:
res=""
else:
# Verify and return if valid response
chksum=self.crcs(res[:-3])
if res[0]=="\xff" and res[1]==chr(cmd) and
chksum==res[-3:-1]:
# Strip off checksum and terminator before returning
return res[:-3]

if len(res)==0:
self.raisecommsexception("using the phonebook")
elif res[0]!="\xff":
self.comm.logdata("Invalid start byte
=",common.datatohexstring(res))
elif chksum!=res[-3:-1]:
self.comm.logdata("Invalid checksum
=",common.datatohexstring(res))

# Return None for pychecker
return None


def sendbrewcommand(self, cmd, data):
# Only repeat if baudrate is low. At higher baudrates timeouts
# frequently occur, so retrying doesn't really help.
if self.comm.ser.ispeed<=38400:
numTimes=2
else:
numTimes=1

# Construct commmand message
msg="\x59"+chr(cmd)+data
msg=self.escape(msg+self.crcs(msg))+self.terminator

# Repeat the indicated number of times
for retry in range(numTimes):
# Send command message to phone
self.comm.write(msg)

# Retrieve response from phone
try:
res=self.unescape(self.comm.readuntil(self.terminator))
except commport.CommTimeout:
res=""
else:
# Verify and return if valid response
chksum=self.crcs(res[:-3])
if res[0]=="\x59" and res[2]=="\x00" and
chksum==res[-3:-1]:
return res

if len(res)==0:
self.raisecommsexception("manipulating the filesystem")
elif res[0]!="\x59":
self.comm.logdata("Invalid start byte
=",common.datatohexstring(res))
elif chksum!=res[-3:-1]:
self.comm.logdata("Invalid checksum
=",common.datatohexstring(res))
elif res[2]!="\x00":
self.comm.logdata("Error code returned
=",common.datatohexstring(res))
err=ord(res[2])
if err==0x1c:
raise BrewNoMoreEntriesException()
if err==0x08:
raise BrewNoSuchDirectoryException()
if err==0x06:
raise BrewNoSuchFileException()
raise BrewCommandException(err)
return res

# Return None for pychecker
return None


class CommConnection:
def __init__(self, logtarget, port, baud=115200, timeout=3,
rtscts=0):
self.logtarget=logtarget
self.port=port
self.log("Connecting to port %s, %d baud, timeout %f,
hardwareflow %d" %
(port, baud, float(timeout), rtscts) )
try:
self.ser=serial.Serial(port, baud, timeout=timeout,
rtscts=rtscts)
except serial.serialutil.SerialException,e:
try:
self.ser=serial.Serial(port, baud, timeout=timeout,
rtscts=rtscts)
except serial.serialutil.SerialException,e:
raise common.CommsOpenFailure(port, e.__str__())
self.log("Connection suceeded")

[Roger Binns]

Post by Tim Lowery
Enclosed are some candidate replacement routines that pertain to serial
communications.

Thank you! I'm currently on a business trip so it will take a few
days to apply. I'll probably release a new test version of 0.6
so people can try out the new routines.

Post by Tim Lowery
Occasionally I'll
encounter "segmentation fault" while fetching the phone data or
expanding file info. The application promptly aborts when this occurs,
and it's hard to figure out why. If you have any ideas about narrowing
this down, let me know.

That is just going to be an issue with the python binary you have.
I believe the Mac version is still 2.3alpha.

Post by Tim Lowery
Also, I noticed that the phone must be set to RS-232 mode when
retrieving version info.

Yup. They are effectively modem (AT) commands. It also has to
be in that mode to use as a modem. This is also why I turn off
version info by default. I may even move it to a seperate
diagnostics menu/dialog.

Roger

[Roger Binns]

I am working on adding the code. The two attempts to open comm port
are in CVS now.

I've got a theory that the other problems are caused by the host computer
changing baud rate too quickly. However I never get comms failures myself
so I can't test it.

Could you please 'import time' at the top of commport.py and
then at the end of setbaudrate add 'time.sleep(1)'
NB this needs to be against the CVS source, not your new routines.

Please tell me if this makes any difference. I would prefer it over
the solution of sending stuff up to twice since that could fail in
other ways (eg if the command being sent is to delete a file but the
first worked but we fudged dealing with the response then we would
send a second command and get a file not existing error).

Roger

[Tim Lowery]

Roger,

I agree with you that a delay in setbaudrate might be necessary, and it
would be preferable to doing retries.

Unfortunately, I can't get the serial communications to fail at the
moment. This is with no delay after the baud rate change, and no
retries in the routines. Usually, it would fail about 1 of 5 times of
GetPhoneData. I guess I'll wait for the weather to change, and try
again.

However, the open comm port started to fail (on both attempts). I
added 'time.sleep(1)' before the 2nd attempt to open, and that seemed
to fix that.

Tim

Post by Roger Binns
I am working on adding the code. The two attempts to open comm port
are in CVS now.
I've got a theory that the other problems are caused by the host
computer
changing baud rate too quickly. However I never get comms failures
myself
so I can't test it.
Could you please 'import time' at the top of commport.py and
then at the end of setbaudrate add 'time.sleep(1)'
NB this needs to be against the CVS source, not your new routines.
Please tell me if this makes any difference. I would prefer it over
the solution of sending stuff up to twice since that could fail in
other ways (eg if the command being sent is to delete a file but the
first worked but we fudged dealing with the response then we would
send a second command and get a file not existing error).
Roger
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bitpim-devel mailing list
https://lists.sourceforge.net/lists/listinfo/bitpim-devel