[BitPim-devel] linux development

[Don Zickus]

Hello,

I was looking to volunteer some time and help this project. One of the
things I was looking at is setting up the hotplug scripts properly on Linux.
Currently on my box they don't work, so I have to change all the file
permissions by hand to use bitpim with my personal account. Unfortunately,
I am not a python programmer, more like a low level kernel programmer.
Reading through the code seem straight forward. The only problem I had was
putting all the pieces together. Maybe its because I am not used to object
oriented programming, but I had trouble visualizing how the lower pieces
worked, namely the comscan stuff.

I understand the comscan logic but not the entire reason. Does bitpim just
scan the /dev/* stuff just to see what is plugged in or does it actually use
those devices for reading/writing? If so, then how does the usb library
fit in to the whole picture.

I have a bunch more questions but I'll wait until I understand this piece a
little better.

Thanks for any help,
Don

[Roger Binns]

Post by Don Zickus
One of the
things I was looking at is setting up the hotplug scripts properly on Linux.
Currently on my box they don't work, so I have to change all the file
permissions by hand to use bitpim with my personal account.

BTW the hotplug scripts have nothing to do with Python. They do fundamentally
work. You have to get the usb ids correct which depend on your phone and/or
cable. We can't install the scripts ourselves since some devices are generic
USB to serial (eg Prolific PL2303 used in many phone as well as other cables)
and there are several different versions of hotplug out there and the various
distro vendors like to move the config files around.

Post by Don Zickus
I am not a python programmer, more like a low level kernel programmer.

That is exactly what we need at the moment. Don't worry about the
Python side. What I do need is C userspace code that will do various
things and needs to run as a non-root user.

Post by Don Zickus
I understand the comscan logic but not the entire reason. Does bitpim just
scan the /dev/* stuff just to see what is plugged in or does it actually use
those devices for reading/writing? If so, then how does the usb library
fit in to the whole picture.

That depends on the phone model and/or cable. Some phones export a USB
modem interface (ACM) and that is what BitPim needs to talk to. BitPim
doesn't have the ACM protocol builtin, so we have to talk to a device
node that implements it.

Other cables are USB to serial (eg PL2303 based) and we have to speak
to the device node that has the relevant driver behind that.

And other phones are composite USB devices and have a secondary diagnostics
interface that must be spoken to. For those phones BitPim uses libusb.

The things we really need are:

- Being able to map from USB devices to the relevant device nodes.
Just how can we tell that USB device X on bus Y happens to be
/dev/ttyACM0? At the moment we just try to open the various
device nodes, but when it fails we can't tell if that is because
there is no device at all, or that there is and there are permissions
issues.

- Notification that there has been a device change (appearance or
removal of any of the relevant device names)

Roger

[Don Zickus]

Roger,

Post by Roger Binns

One of the things I was looking at is setting up the hotplug scripts
properly on Linux. Currently on my box they don't work, so I have to
change all the file permissions by hand to use bitpim with my personal
account.

BTW the hotplug scripts have nothing to do with Python. They do
fundamentally
work. You have to get the usb ids correct which depend on your phone
and/or
cable. We can't install the scripts ourselves since some devices are
generic
USB to serial (eg Prolific PL2303 used in many phone as well as other
cables)
and there are several different versions of hotplug out there and the
various
distro vendors like to move the config files around.

OK. I guess one of the things I wanted to fix (for at least my Fedora
machine) was that the hotplug scripts did not know anything about the /dev
devices. They only dealt with the /proc filesystem. This lead me to my
earlier question about what you actually did with those files. Using the
USB scanner as an example it seemed possible to create symbolic links or
even 'dummy' files for every possible device Bitpim supported (through the
use of a device.usermap). If you just needed an easy way to locate inserted
devices, this seemed like a possible approach. On the other hand if you are
actually opening the devices, it may need some more work. I guess that was
what I was looking to investigate (again for my machine for starters) and
didn't want to step on anyone's toes.

Post by Roger Binns

I am not a python programmer, more like a low level kernel programmer.

That is exactly what we need at the moment. Don't worry about the
Python side. What I do need is C userspace code that will do various
things and needs to run as a non-root user.

Great, please elaborate.

Post by Roger Binns

I understand the comscan logic but not the entire reason. Does bitpim
just scan the /dev/* stuff just to see what is plugged in or does it
actually use those devices for reading/writing? If so, then how does the
usb library fit in to the whole picture.

That depends on the phone model and/or cable. Some phones export a USB
modem interface (ACM) and that is what BitPim needs to talk to. BitPim
doesn't have the ACM protocol builtin, so we have to talk to a device
node that implements it.
Other cables are USB to serial (eg PL2303 based) and we have to speak
to the device node that has the relevant driver behind that.
And other phones are composite USB devices and have a secondary diagnostics
interface that must be spoken to. For those phones BitPim uses libusb.

So you really do need to open the /dev and read/write some data.

Post by Roger Binns
- Being able to map from USB devices to the relevant device nodes.
Just how can we tell that USB device X on bus Y happens to be
/dev/ttyACM0? At the moment we just try to open the various
device nodes, but when it fails we can't tell if that is because
there is no device at all, or that there is and there are permissions
issues.

Ok. I noticed that.

Post by Roger Binns
- Notification that there has been a device change (appearance or
removal of any of the relevant device names)

[Roger Binns]

Incidentally, what phone do you have? The various models and manufacturers
behave differently and it is a bad idea to extrapolate from a sample of one.

Post by Don Zickus
OK. I guess one of the things I wanted to fix (for at least my Fedora
machine) was that the hotplug scripts did not know anything about the /dev
devices.

They should do on other distros. (There is the added complication of whether
or not you are using udev).

Post by Don Zickus
it seemed possible to create symbolic links or
even 'dummy' files for every possible device Bitpim supported (through the
use of a device.usermap).

It isn't as you can't tell whether a USB to serial device is seen and is a
cell phone, a GPS, an old mouse or something else.

Post by Don Zickus
On the other hand if you are
actually opening the devices, it may need some more work.

Yes, we need to open the devices in some cases because the kernel provides
a device driver we need to speak through.

And unfortunately on Linux and Mac, you can't use libusb against a device
for which a driver is installed and they both install the generic ACM
driver, so the workaround of using our own ACM implementation wouldn't
help.

Post by Don Zickus

Post by Roger Binns
That is exactly what we need at the moment. Don't worry about the
Python side. What I do need is C userspace code that will do various
things and needs to run as a non-root user.

Great, please elaborate.

If you have C userspace code that implements functionality we need
then it is trivial for me to glue it into the Python.

Post by Don Zickus
So you really do need to open the /dev and read/write some data.

Yes, for some phones and/or cables. Life would be easier if we
could just stick to libusb and had our own drivers.

[Don Zickus]

Roger,

Post by Roger Binns
Incidentally, what phone do you have? The various models and manufacturers
behave differently and it is a bad idea to extrapolate from a sample of
one.

I am using an LG VX6000 with the FutureDial usb-to-serial cable.

Post by Roger Binns

it seemed possible to create symbolic links or even 'dummy' files for
every possible device Bitpim supported (through the use of a
device.usermap).

It isn't as you can't tell whether a USB to serial device is seen and is a
cell phone, a GPS, an old mouse or something else.

Can't you just probe all the devices that are 'possible' Bitpim targets? I
mean you can still group all possible Bitpim targets in one location and
then probe those targets. If a cellphone is on the other end, great, else
keep probing. Right?

Post by Roger Binns
Yes, we need to open the devices in some cases because the kernel provides
a device driver we need to speak through.

Hmm. Bummer.

Post by Roger Binns
And unfortunately on Linux and Mac, you can't use libusb against a device
for which a driver is installed and they both install the generic ACM
driver, so the workaround of using our own ACM implementation wouldn't
help.

I am starting to understand your nightmares...

Post by Roger Binns
If you have C userspace code that implements functionality we need
then it is trivial for me to glue it into the Python.

Ok.

Post by Roger Binns

So you really do need to open the /dev and read/write some data.

Yes, for some phones and/or cables. Life would be easier if we
could just stick to libusb and had our own drivers.

Right.

Post by Roger Binns
DBus is supposed to solve this, but I couldn't get it to give me
device notifications. I would rather use it directly or whatever
underlying mechanism it uses is.

I thought DBus would take of this too. Huh.

It seems like my best bet is to selfishly get it working on my system first.
Then try to abstract the solution to cover different combinations of
Linux.

Is there an easy way to run a subset of the Bitpim code? It would be great
if I could just run simple tests from the lower layers (ie comscan) just to
see what is working and what is not. Something quick and dirty without the
overhead of a GUI would be nice.

Thanks,
Don

[Roger Binns]

Post by Don Zickus
I am using an LG VX6000 with the FutureDial usb-to-serial cable.

You really should have got a straight USB cable.

Anyway your cable uses a Prolific PL2303 USB to serial chip and this
chip is very widely used. An OS level device driver is needed.

Post by Don Zickus
Can't you just probe all the devices that are 'possible' Bitpim targets?

Yes. We already have code that does that. The problem is what do we do
if "/dev/usb/ttyUSB0" can't be opened in your case. Is that because
of permissions issues, because someone else already has it open, or because
your cable is actually connected to ttyUSB1?

Post by Don Zickus
mean you can still group all possible Bitpim targets in one location and
then probe those targets. If a cellphone is on the other end, great, else
keep probing. Right?

That is exactly what the code currently does, except we can't extract
any meaning from open failures, nor can we map devices seen on the USB
busses to the corresponding device nodes.

Post by Don Zickus
It seems like my best bet is to selfishly get it working on my system first.
Then try to abstract the solution to cover different combinations of
Linux.

Yup.

Post by Don Zickus
Is there an easy way to run a subset of the Bitpim code?

No need. Plug your cable in. Then from user space C code see if you
can work out which USB device is attached to /dev/usb/ttyUSB0. Also
see if you can be notified when the cable gets plugged in.

Lastly I need to be able to "diagnose" open fails on that device. That
means telling the difference between permissions issues, the device not
existing at all, someone else having it open etc.

BTW you can run comscan.py by itself. There is also a usbscan.py.
Run the build script in native/usb if you want to run the latter.

Roger

[Don Zickus]

Roger,

Post by Roger Binns

Post by Don Zickus
I am using an LG VX6000 with the FutureDial usb-to-serial cable.

You really should have got a straight USB cable.

Dumb question: How would that work? I looked at both ends of a straight USB
cable and the end of my phone and I felt like I would be putting a square
peg in a round hole. Did I miss something?

Post by Roger Binns

Post by Don Zickus
Is there an easy way to run a subset of the Bitpim code?

No need. Plug your cable in. Then from user space C code see if you
can work out which USB device is attached to /dev/usb/ttyUSB0. Also
see if you can be notified when the cable gets plugged in.
Lastly I need to be able to "diagnose" open fails on that device. That
means telling the difference between permissions issues, the device not
existing at all, someone else having it open etc.
BTW you can run comscan.py by itself. There is also a usbscan.py.
Run the build script in native/usb if you want to run the latter.

Alright. I'll hack up something in 'C' and see what happens. Give me a few
days (gotta find some time to sit down for a few hours).

Cheers,
Don

[Roger Binns]

Post by Don Zickus

Post by Roger Binns

Post by Don Zickus
I am using an LG VX6000 with the FutureDial usb-to-serial cable.

You really should have got a straight USB cable.

Dumb question: How would that work? I looked at both ends of a straight USB
cable and the end of my phone and I felt like I would be putting a square
peg in a round hole. Did I miss something?

The cables all have USB on one end and the VX6000 connector on the other
end. The VX6000 internals have pins for incoming USB and pins for incoming
serial. The straight USB cable connects to the USB pins. The USB to
serial cable connects to the serial pins. The latter has to have the
convertor chip somewhere, which is usually a blob in the middle of the cable.

Many straight USB cables also have blobs. All charging cables should have
a regulator somewhere. For ease of manufacturing, many have one piece
which is a generic USB to blob, and then a second phone model specific
cable that connects to the blob.

The benefit of the straight USB cable is that communication is a lot quicker
and there is no need to worry about baud rates. Additionally BitPim can
detect the phone a lot quicker since it can examine the USB ids.

Roger