[Bitpim-devel] libusb

[Steven Palm]

I know this is probably too early on to be messing with, but I thought,
what the heck....

I compiled and installed version 1.7 of libusb. I then installed/ran
SWIG to generate the hooks for python, and then tried to run your
usbscan.py script.

First, the build steps for the libusb wrapper:

PYTHONVER=python2.3
INCLUDEDIR=/System/Library/Frameworks/Python.framework/Versions/2.3/
include/$PYTHONVER

swig -python -I/usr/local/include libusb.i

gcc -Wall -O2 -bundle -undefined suppress -flat_namespace -I
$INCLUDEDIR -I /usr/local/include -o _libusb.so libusb_wrap.c
-L/usr/local/lib -lusb -framework Python


SWIG builds out of the box on MacOS X 10.3.

When I run usbscan.py, I get:

usb_set_debug: Setting debugging level to 255 (on)
usb_os_find_busses: Found 003
usb_os_find_busses: Found 003
usb_os_find_busses: Found 003
usb_os_find_busses: Found 002
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 002 on 003
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 001 on 003
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 1004:6000
USB error: could not open device
usb_os_open: 05ac:8006
USB error: could not open device
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 002 on 003
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 001 on 003
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 1004:6000
USB error: could not open device
usb_os_open: 05ac:8006
USB error: could not open device
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 002 on 003
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 001 on 003
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 1004:6000
USB error: could not open device
usb_os_open: 05ac:8006
USB error: could not open device
usb_os_find_devices: Found 002 on 002
usb_os_find_devices: Found 001 on 002
usb_os_open: 05ac:8005
USB error: could not open device
usb_os_open: 05ac:8203
USB error: could not open device
usb_os_open: 05ac:8006
USB error: could not open device
usb_open(<C usb_device instance at _02848800_p_usb_device>)=None
Traceback (most recent call last):
File "usbscan.py", line 70, in ?
res=usbscan()
File "usbscan.py", line 28, in usbscan
for device in bus.devices():
File "/Users/n9yty/my_cvs/bitpim/bitpim/native/usb/usb.py", line 43,
in devices
yield USBDevice(dev)
File "/Users/n9yty/my_cvs/bitpim/bitpim/native/usb/usb.py", line 56,
in __init__
raise USBException()
native.usb.usb.USBException: could not open device

I don't know if this helps at all, but this is the system view of the
USB bus:

USB:

USB Bus:

Vendor Name: Apple Computer, Inc.
Product ID: 32773 ($8005)
Speed: Up to 12 Mb/sec
Bus Power (mA): 500

Bluetooth HCI:

Speed: Up to 12 Mb/sec
Product ID: 33283 ($8203)
Bus Power (mA): 500
Vendor ID: 1452

USB Bus:

Vendor Name: Apple Computer, Inc.
Speed: Up to 12 Mb/sec
Product ID: 32773 ($8005)
Bus Power (mA): 500

USB Bus:

Vendor Name: Apple Computer, Inc.
Product ID: 32773 ($8005)
Speed: Up to 12 Mb/sec
Bus Power (mA): 500

Qualcomm CDMA Technologies MSM:

Vendor Name: LG CDMA USB Modem
Speed: Up to 12 Mb/sec
Product ID: 24576 ($6000)
Bus Power (mA): 500

USB High-Speed Bus:

Vendor Name: Apple Computer, Inc.
Speed: Up to 480 Mb/sec
Product ID: 32774 ($8006)
Bus Power (mA): 500

Again, if this is too early to be messing with, tell me to go away for
now. ;-)

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--

[Steven Palm]

Post by Steven Palm
I compiled and installed version 1.7 of libusb. I then installed/ran
SWIG to generate the hooks for python, and then tried to run your
usbscan.py script.

I just compiled the CVS tree version of libusb, and the output differs
slightly:

[Roger Binns]

Post by Steven Palm

Post by Steven Palm
usb_set_debug: Setting debugging level to 255 (on)
usb_os_find_busses: Found 003
usb_os_find_busses: Found 003
usb_os_find_busses: Found 003
usb_os_find_busses: Found 002
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 002 on 003
usb_os_find_devices: Found 001 on 003
usb_os_find_devices: Found 001 on 003

That is all normal.

Post by Steven Palm

Post by Steven Palm
usb_os_open: 05ac:8005
USB error: could not open device

That isn't. My complete output on Linux is at the bottom.

However looking at the code, I only need to open the device
to get vendor and product strings (not the same as the
integer ids) and to actually open the bulk interfaces
later on.

I will look into making that change in the code, although
it won't do any good for you anyway.

Post by Steven Palm
usb_os_open: 1004:6000
USB error: usb_os_open: Another process already has exclusive access to
the device.

That is your phone. The libusb api has you claiming the
interfaces seperately from the device as a whole. On Linux
you can't claim an interface if there is a device driver
attached, but the acm driver only attaches to interface
0 & 1 (interrupt and data for the modem) and leaves the
diagnostics interface (2) seperate. I don't know how you
get around it.

Some places you should look at are projects on SourceForge
for scanners and digital cameras. They usually use libusb
on Linux, Windows and Mac.

Here is my complete output:

rb.com rogerb /space/bitpim/native/usb > python usb.py
usb_set_debug: Setting debugging level to 255 (on)
usb_os_init: Found USB VFS at /proc/bus/usb
usb_find_busses: Skipping non bus directory devices
usb_find_busses: Skipping non bus directory drivers
usb_find_busses: Found 002
usb_find_busses: Found 001
usb_find_devices_on_bus: Found 001 on 001
usb_find_devices_on_bus: Found 002 on 001
usb_find_devices_on_bus: Found 003 on 001
usb_find_devices_on_bus: Found 010 on 001
skipped 4 class/vendor specific interface descriptors
usb_find_devices_on_bus: Found 001 on 002
0 busses, 0 devices
001
0/0 001
class USB_CLASS_HUB subclass 0 protocol 0
interface number 0
class USB_CLASS_HUB subclass 0 protocol 0
endpointaddress 0x1
TYPE_INTERRUPT


451/2046 002
class USB_CLASS_HUB subclass 0 protocol 0
interface number 0
class USB_CLASS_HUB subclass 0 protocol 0
endpointaddress 0x1
TYPE_INTERRUPT


4b8/5 003
class USB_CLASS_PER_INTERFACE subclass 0 protocol 0
interface number 0
class USB_CLASS_PRINTER subclass 1 protocol 2
endpointaddress 0x1
TYPE_BULK OUT
endpointaddress 0x2
TYPE_BULK IN


1004/6000 010
class USB_CLASS_COMM subclass 0 protocol 0
interface number 0
class USB_CLASS_COMM subclass 2 protocol 1
endpointaddress 0x1
TYPE_INTERRUPT

interface number 1
class USB_CLASS_DATA subclass 0 protocol 0
endpointaddress 0xa
TYPE_BULK IN
endpointaddress 0xb
TYPE_BULK OUT

interface number 2
class USB_CLASS_VENDOR_SPEC subclass 255 protocol 0
endpointaddress 0x3
TYPE_BULK IN
endpointaddress 0x6
TYPE_BULK OUT



002
0/0 001
class USB_CLASS_HUB subclass 0 protocol 0
interface number 0
class USB_CLASS_HUB subclass 0 protocol 0
endpointaddress 0x1
TYPE_INTERRUPT


Roger

[Steven Palm]

Post by Roger Binns
That is your phone. The libusb api has you claiming the
interfaces seperately from the device as a whole. On Linux
you can't claim an interface if there is a device driver
attached, but the acm driver only attaches to interface
0 & 1 (interrupt and data for the modem) and leaves the
diagnostics interface (2) seperate. I don't know how you
get around it.

Let me get this straight, because I think I'm reading that both
ways....

Either you *CAN* grab the interfaces separately from the device as a
whole, thus allowing the kernel driver to handle the serial_com
interfaces and leaving the vendor_specific interface for you, *OR* you
can NOT grab them separately in which case I don't know how you'd get
it to work. ;-)

(from another e-mail, you write)

Post by Roger Binns
that). The second bit looks to be the driver it attached but it
doesn't
look like it gave you a device name.

Not all USB devices on MacOS are actually assigned a filesystem device
name. They aren't accessed by that anyway... If you look at the libusb
code, darwin.c in particular, you'll see the way to enlightenment on
MacOS by stepping through IOKit for enumerated devices and interfaces
on them. The trouble is, as far as I can tell, that if MacOS X assigns
a driver to the device you can not open it to enumerate the interfaces.

I'm working on it... ;^)

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--

[Steven Palm]

Post by Steven Palm
Either you *CAN* grab the interfaces separately from the device as a
whole, thus allowing the kernel driver to handle the serial_com
interfaces and leaving the vendor_specific interface for you, *OR* you
can NOT grab them separately in which case I don't know how you'd get
it to work. ;-)

Okay, my head is starting to settle down a little bit... Thanks to
someone at Apple, I am a little closer here. I was able to throw
together some sample code and actually open the data interface on the
phone... But, the question is, how to make this method compatible with
libusb and bitpim....

Due to the way composite devices work on MacOS X, at least CDC devices
in particular, you can NOT open the device, as there is no way to
wrestle control away from the kernel extension that has it open. The
current logic being used seems to want to walk the tree looking for a
match with vendor and product code on a device, and then get it's
second interface, but since you cannot open the device this whole
approach falls apart.

On MacOS X for this device, you *CAN* get to the interface in
question, but you actually have to go directly for the interface by
vendor, product, configuration and interface number. This shouldn't
pose a massively huge problem, because you know the
vendor/product/interface number already, and the code appeared to only
be using the first configuration so this is fine as well.

The question becomes, how to hack libusb so it will work or, if that
falls through, write a separate access routine for the Mac, at least
the 'open/init' bit.

What do you think?

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--

[Roger Binns]

Post by Steven Palm
The
current logic being used seems to want to walk the tree looking for a
match with vendor and product code on a device, and then get it's
second interface, but since you cannot open the device this whole
approach falls apart.

That is what the native/usb/usb.py code does as a test, but isn't
what bitpim proper does. BitPim invokes usbscan.py in the main
code to get a list of all USB devices and names them
usb::bus::device::interface

comdiagnose then picks out what devices from the output of
comscan.py and usbscan.py are likely to be phones or USB
to serial cables.

The reason for using names is so that particular devices can
be selected. As a worst case example, here is usbscan output
for my machine with a USB to serial cable, a VX4400 and a
VX6000 plugged in.

UsbScan 7 December 2003

usb::001::016::0:
active: 1 available: 0
description: USB Device - Vendor 0x4b8 Product 0x5 (Interface 0) libusb: 1
usb-interface: 0 usb-product: 5 usb-vendor: 1208

usb::001::020::0:
active: 1 available: 0
description: USB Device - Vendor 0x67b Product 0x2303 (Interface 0)
libusb: 1 usb-interface: 0 usb-product: 8963 usb-vendor: 1659

usb::001::022::1:
active: 1 available: 0
description: USB Device - Vendor 0x1004 Product 0x6000 (Interface 1)
libusb: 1 usb-interface: 1 usb-product: 24576 usb-vendor: 4100

usb::001::022::2:
active: 1 available: 1
description: USB Device - Vendor 0x1004 Product 0x6000 (Interface 2)
libusb: 1 usb-interface: 2 usb-product: 24576 usb-vendor: 4100

usb::001::021::1:
active: 1 available: 0
description: USB Device - Vendor 0x1004 Product 0x6000 (Interface 1)
libusb: 1 usb-interface: 1 usb-product: 24576 usb-vendor: 4100

usb::001::021::2:
active: 1 available: 1
description: USB Device - Vendor 0x1004 Product 0x6000 (Interface 2)
libusb: 1 usb-interface: 2 usb-product: 24576 usb-vendor: 4100

Post by Steven Palm
On MacOS X for this device, you *CAN* get to the interface in
question, but you actually have to go directly for the interface by
vendor, product, configuration and interface number.

What if there are multiple devices with the same details?

Post by Steven Palm
The question becomes, how to hack libusb so it will work or, if that
falls through, write a separate access routine for the Mac, at least
the 'open/init' bit.

You have to call usb_open in order to call usb_claim_interface in
libusb. The actual opening of a device happens in _openusb in
commport.py.

You can build a seperate interface for USB if you want to instead
of using libusb. You can see another example in native/wab
where I actually wrote my own code for Python interfacing.

Roger

[Steven Palm]

Post by Roger Binns
That is what the native/usb/usb.py code does as a test, but isn't
what bitpim proper does. BitPim invokes usbscan.py in the main
code to get a list of all USB devices and names them
usb::bus::device::interface

What a shame, then, that MacOS X sort of gets in the way of this.
Well, perhaps not, because if you can't use the device then what is the
point of allowing it to show up? In this case, I mean the other
interfaces which are already spoken for by the kernel drivers... I am
able to get a list of the busses and vendor/products of the devices on
the bus, I just can't open the device if it's busy. I suppose I could
*try* to open a whole range of interfaces on the device, say, 0-32, and
save a list of the ones that could be opened.

Post by Roger Binns
comdiagnose then picks out what devices from the output of comscan.py
and usbscan.py are likely to be phones or USB to serial cables.

Well, USB-to-serial cables would be mapped (at least on MacOS X) to
/dev/cu.* devices (most likely), and then could be handled outside of
the USB stuff.

Post by Roger Binns

Post by Steven Palm
On MacOS X for this device, you *CAN* get to the interface in
question, but you actually have to go directly for the interface by
vendor, product, configuration and interface number.

What if there are multiple devices with the same details?

You actually get an iterator object that would cover all matching
interfaces for your specification.

Post by Roger Binns
You can build a seperate interface for USB if you want to instead
of using libusb. You can see another example in native/wab
where I actually wrote my own code for Python interfacing.

Okay, this will bear more thinking things through, then.

I see that you have a usbdb list of phones supported and their
vendor/product/config/interface information. This would be once source
to use for just trying to see if the interfaces could be opened, and if
so, put them in the list (for the Mac, at least). I see you have some
USB->serial cables there with no interface listed. Do these connect as
USB or through a device entry?

At any rate, I will play around with a few things and get back to you.
Unless I get it working at all, there's not much point in haggling over
how to fit it into the project. :-)

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--

[Roger Binns]

Post by Steven Palm
What a shame, then, that MacOS X sort of gets in the way of this.
Well, perhaps not, because if you can't use the device then what is the
point of allowing it to show up?

The code is in two pieces. One piece (comscan and usbscan) finds all
devices. The second piece (comdiagnose) then makes an intelligent
decision about which of the devices are likely to be the phone, and
prioritises them.

At the moment comdiagnose has the USB ids in its code. That information
will be moving to the profile class for each phone.

Post by Steven Palm
In this case, I mean the other
interfaces which are already spoken for by the kernel drivers... I am
able to get a list of the busses and vendor/products of the devices on
the bus, I just can't open the device if it's busy. I suppose I could
*try* to open a whole range of interfaces on the device, say, 0-32, and
save a list of the ones that could be opened.

comscan and usbscan do include information about whether the particular
port/device is available. There are both 'active' and 'available'
flags. 'active' would be false if the driver etc existed but the cable/
device was unplugged. 'available' means bitpim could use it.
The simple test in comscan is to try to open the device, and something
similar is done in usb scan.

Post by Steven Palm
Well, USB-to-serial cables would be mapped (at least on MacOS X) to
/dev/cu.* devices (most likely), and then could be handled outside of
the USB stuff.

And under Windows they are mapped to COM ports, and under Linux to
/dev/usb/ttyUSB0..

However the USB information is still useful. For example under Windows
the USB to serial cable has a hardwareinstance value of
...&VID_1234&PID_1234

That is how I can tell that particular com port is a USB to serial
cable (and hence could have a phone on the end) as opposed to a motherboard
serial port which definitely doesn't have a phone on it.

Post by Steven Palm

Post by Steven Palm
On MacOS X for this device, you *CAN* get to the interface in
question, but you actually have to go directly for the interface by
vendor, product, configuration and interface number.

The good news is that almost every device in existence only has 1
configuration and the configuration value is 1.

Post by Steven Palm
I see you have some
USB->serial cables there with no interface listed. Do these connect as
USB or through a device entry?

They are connected through the Windows COM port name, not directly via
libusb. As mentioned above, I can use the usb ids to figure out what
the com port actually is.

Post by Steven Palm
At any rate, I will play around with a few things and get back to you.
Unless I get it working at all, there's not much point in haggling over
how to fit it into the project. :-)

Yup. Get some horrible disgusting hack working and then take it from
there :-)

Roger

[Roger Binns]

Here is an example of comscan on my machine. [I would
have included this in the last message, except I found
a bug in comscan that I have now fixed]

COM1:
active: True available: True description: Communications Port (COM1)
driverdate: (2001, 7, 1) driverdescription: Communications Port
driverprovider: Microsoft driverversion: 5.1.2600.0
hardwareinstance: ACPI\PNP0501\1

COM2:
active: True available: True description: Communications Port (COM2)
driverdate: (2001, 7, 1) driverdescription: Communications Port
driverprovider: Microsoft driverversion: 5.1.2600.0
hardwareinstance: ACPI\PNP0501\2

COM5:
active: False available: False
description: Prolific USB-to-Serial Comm Port (COM5)
driverdate: (2002, 4, 9)
driverdescription: Prolific USB-to-Serial Comm Port
driverprovider: Prolific driverversion: 1.5.0.0
hardwareinstance: USB\VID_067B&PID_2303\6&32D8FA8&0&3

COM6:
active: True available: True
description: Prolific USB-to-Serial Comm Port (COM6)
driverdate: (2002, 4, 9)
driverdescription: Prolific USB-to-Serial Comm Port
driverprovider: Prolific driverversion: 1.5.0.0
hardwareinstance: USB\VID_067B&PID_2303\6&32D8FA8&0&4

COM1 and COM2 are the onboard ports. COM5 and COM6 are the USB to serial
ports. Note how I can detect the USB info in the hardwareinstance.
The reason why it lists both COM5 and 6 is that I have a hub
and it assigns each position in the hub a different hardware
instance. I only have one cable so it doesn't matter, but
if I had 4 cables it would be needed to tell the difference.
That is also why COM5 is neither active nor available since
it isn't plugged in.

Roger

[Steven Palm]

If I remove the AppleUSBCDCDriver.kext, I get the following:

usb_set_debug: Setting debugging level to 255 (on)
usb_os_find_busses: Found 026
usb_os_find_busses: Found 027
usb_os_find_busses: Found 059
usb_os_find_busses: Found 091
usb_os_find_devices: Found 001 on 026
usb_os_find_devices: Found 002 on 026
usb_os_open: 05ac:8203
USB error: usb_os_open: Another process already has exclusive access to
the device.
usb_os_open: 05ac:8005
USB error: usb_os_open: Another process already has exclusive access to
the device.
usb_os_find_devices: Found 001 on 027
usb_os_open: 05ac:8005
USB error: usb_os_open: Another process already has exclusive access to
the device.
usb_os_find_devices: Found 001 on 059
usb_os_find_devices: Found 002 on 059
usb_os_open: 1004:6000
usb_os_open: device opened
usb_control_msg: 128 6 512 0 0xbffff5c8 8 1000
usb_control_msg: 128 6 512 0 0x510440 90 1000
skipped 4 class/vendor specific interface descriptors
usb_os_close: 1004:6000
usb_os_open: 05ac:8005
USB error: usb_os_open: Another process already has exclusive access to
the device.
usb_os_find_devices: Found 001 on 091
usb_os_open: 05ac:8006
USB error: usb_os_open: Another process already has exclusive access to
the device.
4 busses, 6 devices
026
usb_os_open: 05ac:8005
USB error: usb_os_open: Another process already has exclusive access to
the device.
usb_open(<C usb_device instance at _02848800_p_usb_device>)=None
Traceback (most recent call last):
File "usb.py", line 300, in ?
for device in bus.devices():
File "usb.py", line 43, in devices
yield USBDevice(dev)
File "usb.py", line 56, in __init__
raise USBException()
__main__.USBException: usb_os_open: Another process already has
exclusive access to the device.

So, while it no longer gets the "device in use" error when trying to
open the phone, it's not quite getting anywhere useful either. That's
my last shot for the night... At least AppleUSBCDCDriver.kext is in the
Darwin open source project so I can hopefully check it out and see what
is going on.

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--