Discussion:
[Bitpim-devel] Mac bundle procedure....
Steven Palm
2003-12-09 04:32:25 UTC
Permalink
Okay, I've managed to get a Macintosh bundle built that runs on a
system other than my own (although only tested on two Panther (10.3)
systems).
Looking at the bitpim source, my guess is that this line is causing
resourcedirectory=os.path.abspath(os.path.join(os.path.dirname(sys.argv
[0]), 'resources'))
Using sys.argv like this can cause problems on OS X, which sends some
extra args in when launched from the Finder. I'd suggest trying it
resourcedirectory=os.path.abspath(os.path.join(sys.path[0]),
'resources'))
That is how I determine the root directory of my program, which runs
on Windows, Linux and Mac.
I made the changes outlined above in guihelper.py and bpaudio.py, the
only places I found them.

Also, here is the macbuild.py script I used (stealing enough from
distbuild.py to make it work):

import bundlebuilder, os, glob
packageroot = "/Users/n9yty/my_cvs/bitpim/bitpim"
myapp = bundlebuilder.AppBuilder(verbosity=1)
myapp.mainprogram = os.path.join(packageroot, "bp.py")
myapp.standalone = 1
myapp.name = "bitpim"
myapp.strip = 1
myapp.includePackages.append("sha")
exts=[ '*.xy', '*.png', '*.ttf', '*.wav', '*.jpg', '*.css', '*.htb',
'*.pdc' ]
for wildcard in exts:
for file in glob.glob(os.path.join(packageroot, 'resources',
wildcard)):
myapp.files.append( (file, os.path.join("Contents",
"Resources", 'resources', os.path.basename(file))) )
for file in glob.glob(os.path.join(packageroot, 'com_*.py')):
myapp.resources.append(file)
myapp.libs.append("/usr/local/lib/libwx_mac-2.4.0.dylib")
myapp.libs.append("/usr/local/lib/libwx_mac-2.4.0.rsrc")
myapp.libs.append("/usr/local/lib/libwx_mac_gl-2.4.0.dylib")
myapp.setup()
myapp.build()

There is, undoubtedly, a cleaner way, and this could likely be
integrated into distbuild.py. The odd line in there (for me) is the
myapp.files.append() bit which takes a tuple specifying the file to
add, and the path inside the MacOS X bundle to put it. I'm sure that
makes perfect sense to someone used to tuples. :^)

The downside is that since this bundles the libwx_mac_* and wxPython
shared libraries as well as the bitpim stuff, it weights in at just
over 16MB. But it *IS* a drag-n-drop install application.

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--
Roger Binns
2003-12-09 07:02:28 UTC
Permalink
Post by Steven Palm
resourcedirectory=os.path.abspath(os.path.join(sys.path[0]),
'resources'))
I have committed that change into BitPim 0.7 (HEAD) as well as
the 0.6 branch. I did confirm it also works on Windows and
Linux.
Post by Steven Palm
I made the changes outlined above in guihelper.py and bpaudio.py, the
only places I found them.
bpaudio was duplicating guihelper so I redirected the call. It wouldn't
have actually made any difference in the real world as bpaudio
depends on a Windows binary from Qualcomm (who won't make it
available on other platforms or as source).
Post by Steven Palm
Also, here is the macbuild.py script I used (stealing enough from
I like to integrate it into makedist.py as that already deals
with the platform issues.
Post by Steven Palm
myapp.includePackages.append("sha")
What does that do? It should automatically pick up the usage
of the sha module.
Post by Steven Palm
myapp.resources.append(file)
Why do those end up as resources? Wouldn't they be part
of includePackages?
Post by Steven Palm
myapp.libs.append("/usr/local/lib/libwx_mac-2.4.0.dylib")
myapp.libs.append("/usr/local/lib/libwx_mac-2.4.0.rsrc")
myapp.libs.append("/usr/local/lib/libwx_mac_gl-2.4.0.dylib")
Is there anyway to automate finding those libraries?

Also how do you direct where output goes. For Linux and Windows,
your cwd needs to be the top of the bitpim tree and you run
makedist.py. The output goes to a directory named dist. I
would prefer to do the same thing for the Mac.
Post by Steven Palm
There is, undoubtedly, a cleaner way, and this could likely be
integrated into distbuild.py.
I'd be quite happy to put in something nasty, and then work on
cleaning it up.
Post by Steven Palm
I'm sure that
makes perfect sense to someone used to tuples. :^)
You will be assimilated :-) I would also appreciate a directory
listing of the results.
Post by Steven Palm
The downside is that since this bundles the libwx_mac_* and wxPython
shared libraries as well as the bitpim stuff, it weights in at just
over 16MB. But it *IS* a drag-n-drop install application.
I very much like your last statement. BTW does it include the Python
interpretter as well? Below is what it looks like on Windows and
Linux.

Windows:

Note that Windows also includes a seperate binary for audio format
conversion which is 1.3MB.

5,343,710 bitpim-0.7-test1-setup.exe

09/09/2003 03:18 PM 437 bitpim.css
12/08/2003 10:16 PM 908,469 bitpim.exe
08/20/2003 01:26 PM 655 bitpim.exe.manifest
10/01/2003 12:40 PM 61,440 calendarc.pyd
10/02/2003 08:03 PM 45,116 datetime.pyd
10/01/2003 12:40 PM 286,720 gizmosc.pyd
10/01/2003 12:40 PM 299,008 gridc.pyd
10/01/2003 12:40 PM 36,864 helpc.pyd
10/01/2003 12:40 PM 196,608 htmlc.pyd
10/02/2003 08:03 PM 45,114 parser.pyd
10/02/2003 08:02 PM 974,908 python23.dll
09/02/2003 11:01 AM 86,084 PyWinTypes23.dll
12/08/2003 10:16 PM <DIR> resources
10/02/2003 08:04 PM 20,538 select.pyd
09/02/2003 11:01 AM 61,497 win32api.pyd
09/02/2003 11:01 AM 28,731 win32event.pyd
09/02/2003 11:01 AM 73,786 win32file.pyd
10/01/2003 12:40 PM 2,240,512 wxc.pyd
10/01/2003 10:43 AM 3,239,936 wxmsw24h.dll
10/02/2003 08:04 PM 61,496 zlib.pyd
12/08/2003 09:03 PM 137,934 _libusb.dll
10/02/2003 08:02 PM 49,211 _socket.pyd
10/02/2003 08:03 PM 57,400 _sre.pyd
10/02/2003 08:03 PM 495,616 _ssl.pyd
10/02/2003 08:04 PM 36,864 _winreg.pyd

The actual .py source files are compiled to .pyo, stuck in a zip
and placed in bitpim.exe which invokes the interpretter. Expanded,
the main directory adds up to 9.4MB with resources adding another
2MB.

Linux:

5,066,014 bitpim-0.7_test1-0.i386.rpm

-r-xr-xr-x 1 root root 57186 Feb 24 2003 binascii.so
-rw-r--r-- 1 root root 432 Dec 8 22:35 bitpim.css
-rwxr-xr-x 1 root root 4790871 Dec 8 22:35 bp
-r-xr-xr-x 1 root root 57901 Feb 24 2003 cStringIO.so
-r-xr-xr-x 1 root root 48588 Feb 24 2003 fcntlmodule.so
-rwxr-xr-x 1 root root 21468 Feb 24 2003 _hotshot.so
-rwxr-xr-x 1 root root 6038316 Dec 8 22:36 libwx_gtkd-2.4.so
-r-xr-xr-x 1 root root 58568 Feb 24 2003 _localemodule.so
-r-xr-xr-x 1 root root 51701 Feb 24 2003 mathmodule.so
-rwxr-xr-x 1 root root 872408 Dec 8 20:39 native.usb._libusb.so
-r-xr-xr-x 1 root root 55633 Feb 24 2003 operator.so
-r-xr-xr-x 1 root root 95610 Feb 24 2003 parsermodule.so
-rwxr-xr-x 1 root root 39664 Feb 24 2003 pcre.so
-r-xr-xr-x 1 root root 40272 Feb 24 2003 pwdmodule.so
-r-xr-xr-x 1 root root 64366 Feb 24 2003 readline.so
drwxr-xr-x 2 root root 4096 Dec 8 22:42 resources
-r-xr-xr-x 1 root root 51305 Feb 24 2003 selectmodule.so
-r-xr-xr-x 1 root root 48847 Feb 24 2003 shamodule.so
-r-xr-xr-x 1 root root 156636 Feb 24 2003 _socketmodule.so
-rwxr-xr-x 1 root root 22344 Feb 24 2003 strop.so
-r-xr-xr-x 1 root root 68162 Feb 24 2003 structmodule.so
-r-xr-xr-x 1 root root 52464 Feb 24 2003 termios.so
-r-xr-xr-x 1 root root 54559 Feb 24 2003 timemodule.so
-r-xr-xr-x 1 root root 40105 Feb 24 2003 _weakref.so
-rwxr-xr-x 1 root root 49484 Dec 8 22:36 wxPython.calendarc.so
-rwxr-xr-x 1 root root 396488 Dec 8 22:36 wxPython.gizmosc.so
-rwxr-xr-x 1 root root 308272 Dec 8 22:36 wxPython.gridc.so
-rwxr-xr-x 1 root root 25468 Dec 8 22:36 wxPython.helpc.so
-rwxr-xr-x 1 root root 198808 Dec 8 22:36 wxPython.htmlc.so
-rwxr-xr-x 1 root root 2367852 Dec 8 22:36 wxPython.wxc.so
-r-xr-xr-x 1 root root 57454 Feb 24 2003 zlibmodule.so

The is what gets installed. Total size including the resources subdirectory
is 16.4MB. The 'bp' executable contains the Python interpretter
with the .py files compiled to .pyo and stuck on the end.

Roger
Steven Palm
2003-12-09 14:01:50 UTC
Permalink
Post by Roger Binns
Post by Steven Palm
myapp.includePackages.append("sha")
What does that do? It should automatically pick up the usage
of the sha module.
The sha module is being imported by one of the com_*.py modules. Since
they are dynamically loaded at runtime, the bundle procedure cannot
find the dependancy. It is only looking at import statements for the
main script and anything imported below that via import statements.
Post by Roger Binns
Post by Steven Palm
myapp.resources.append(file)
Why do those end up as resources? Wouldn't they be part
of includePackages?
When I did that, they ended up inside a "Modules.zip" file, and they
could not be found when needed. I will play around with this a little
bit more, but at least it works "as is" and can be used for now with
tweaks later.
Post by Roger Binns
Post by Steven Palm
myapp.libs.append("/usr/local/lib/libwx_mac-2.4.0.dylib")
myapp.libs.append("/usr/local/lib/libwx_mac-2.4.0.rsrc")
myapp.libs.append("/usr/local/lib/libwx_mac_gl-2.4.0.dylib")
Is there anyway to automate finding those libraries?
Roger Binns
2003-12-10 03:04:10 UTC
Permalink
Post by Steven Palm
The sha module is being imported by one of the com_*.py modules. Since
they are dynamically loaded at runtime, the bundle procedure cannot
find the dependancy. It is only looking at import statements for the
main script and anything imported below that via import statements.
Ok. I could also add a redundant import sha in another module.
Post by Steven Palm
-rw-r--r-- 1 n9yty admin 1093458 9 Dec 07:53 Modules.zip
-rw-r--r-- 1 n9yty admin 521 15 Nov 06:16 bp.py
-rw-r--r-- 1 n9yty admin 15059 8 Dec 09:07 com_brew.py
-rw-r--r-- 1 n9yty admin 4390 15 Nov 06:16 com_lg.py
-rw-r--r-- 1 n9yty admin 4791 6 Dec 20:52 com_lgtm520.py
-rw-r--r-- 1 n9yty admin 30153 7 Dec 00:12 com_lgvx4400.py
I assume that Modules.zip just contains the .py files. I also
assume this directory structure isn't writable when deployed.
On Linux and Windows the .py files are compiled to .pyo files
(optimised). This means that all the doc strings are stripped
out which makes them considerably smaller.
Post by Steven Palm
14M dist
What size does the final distributable work out as. IIRC you
make a dimg (disk image) which is a compressed loopback filesystem.
Post by Steven Palm
So here, again, are the needed statements to make it work (output goes
in 'dist')
BTW I assume that sys.platform returns 'darwin'.

Also how do you encode the version numbers of BitPim itself. On
Windows the installer has a unique id associated with BitPim
and uninstalls an existing version first before putting a new
one on. The version number is in the installer, and part
of the setup.exe filename. The default install location is
c:\Program Files\bitpim

On Linux, the files end up in /usr/lib/bitpim-version. There
is wrapper script as /usr/bin/bitpim that execs the main binary
out of /usr/lib/bitpim-version.

In both cases you could have multiple versions installed, and
they would work, but noone does that in practise (and you would
have to put a little effort in to convince the installers you
really want to do that).

I have integrated the code into makedist.py. Thanks for your
continuing contributions on this.

Roger
Steven Palm
2003-12-10 03:34:15 UTC
Permalink
Post by Roger Binns
I assume that Modules.zip just contains the .py files. I also
assume this directory structure isn't writable when deployed.
On Linux and Windows the .py files are compiled to .pyo files
(optimised). This means that all the doc strings are stripped
out which makes them considerably smaller.
Somewhat. It stores the .pyc files, not only for the bitpim stuff, but
any associated items. Here is a list that has been put into it here:

Archive: Modules.zip
testing: Carbon/__init__.pyc OK
testing: Carbon/AE.pyc OK
testing: Carbon/Appearance.pyc OK
testing: Carbon/AppleEvents.pyc OK
testing: Carbon/CarbonEvents.pyc OK
testing: Carbon/ControlAccessor.pyc OK
testing: Carbon/Controls.pyc OK
testing: Carbon/Ctl.pyc OK
testing: Carbon/Dialogs.pyc OK
testing: Carbon/Dlg.pyc OK
testing: Carbon/Dragconst.pyc OK
testing: Carbon/Events.pyc OK
testing: Carbon/Evt.pyc OK
testing: Carbon/File.pyc OK
testing: Carbon/Files.pyc OK
testing: Carbon/Folder.pyc OK
testing: Carbon/Folders.pyc OK
testing: Carbon/Menu.pyc OK
testing: Carbon/Qd.pyc OK
testing: Carbon/QuickDraw.pyc OK
testing: Carbon/Res.pyc OK
testing: Carbon/TextEdit.pyc OK
testing: Carbon/Win.pyc OK
testing: Carbon/Windows.pyc OK
testing: DSV/__init__.pyc OK
testing: DSV/DSV.pyc OK
testing: EasyDialogs.pyc OK
testing: FCNTL.pyc OK
testing: Finder/__init__.pyc OK
testing: Finder/Containers_and_folders.pyc OK
testing: Finder/Enumerations.pyc OK
testing: Finder/Files.pyc OK
testing: Finder/Finder_Basics.pyc OK
testing: Finder/Finder_items.pyc OK
testing: Finder/Legacy_suite.pyc OK
testing: Finder/Standard_Suite.pyc OK
testing: Finder/Type_Definitions.pyc OK
testing: Finder/Window_classes.pyc OK
testing: MacOS.pyc OK
testing: Nav.pyc OK
testing: Queue.pyc OK
testing: StdSuites/__init__.pyc OK
testing: StdSuites/AppleScript_Suite.pyc OK
testing: StdSuites/Macintosh_Connectivity_Clas.pyc OK
testing: StdSuites/QuickDraw_Graphics_Suite.pyc OK
testing: StdSuites/QuickDraw_Graphics_Suppleme.pyc OK
testing: StdSuites/Required_Suite.pyc OK
testing: StdSuites/Standard_Suite.pyc OK
testing: StdSuites/Table_Suite.pyc OK
testing: StdSuites/Text_Suite.pyc OK
testing: StdSuites/Type_Names_Suite.pyc OK
testing: StringIO.pyc OK
testing: TERMIOS.pyc OK
testing: UserDict.pyc OK
testing: _AE.pyc OK
testing: _Ctl.pyc OK
testing: _Dlg.pyc OK
testing: _Evt.pyc OK
testing: _File.pyc OK
testing: _Folder.pyc OK
testing: _Menu.pyc OK
testing: _Qd.pyc OK
testing: _Res.pyc OK
testing: _Win.pyc OK
testing: __future__.pyc OK
testing: __main__.pyc OK
testing: _builtinSuites/__init__.pyc OK
testing: _builtinSuites/builtin_Suite.pyc OK
testing: _hotshot.pyc OK
testing: _locale.pyc OK
testing: _random.pyc OK
testing: _socket.pyc OK
testing: _ssl.pyc OK
testing: _weakref.pyc OK
testing: aepack.pyc OK
testing: aetools.pyc OK
testing: aetypes.pyc OK
testing: analyser.pyc OK
testing: applesingle.pyc OK
testing: array.pyc OK
testing: atexit.pyc OK
testing: bdb.pyc OK
testing: binascii.pyc OK
testing: bpaudio.pyc OK
testing: brewcompressedimage.pyc OK
testing: cStringIO.pyc OK
testing: calendar.pyc OK
testing: calendarcontrol.pyc OK
testing: cmd.pyc OK
testing: comdiagnose.pyc OK
testing: common.pyc OK
testing: commport.pyc OK
testing: comscan.pyc OK
testing: copy.pyc OK
testing: copy_reg.pyc OK
testing: datetime.pyc OK
testing: difflib.pyc OK
testing: dis.pyc OK
testing: doctest.pyc OK
testing: dummy_thread.pyc OK
testing: fcntl.pyc OK
testing: fixedscrolledpanel.pyc OK
testing: fnmatch.pyc OK
testing: getopt.pyc OK
testing: glob.pyc OK
testing: gui.pyc OK
testing: guihelper.pyc OK
testing: guiwidgets.pyc OK
testing: helpids.pyc OK
testing: hexeditor.pyc OK
testing: hotshot/__init__.pyc OK
testing: hotshot/log.pyc OK
testing: hotshot/stats.pyc OK
testing: ic.pyc OK
testing: icglue.pyc OK
testing: importexport.pyc OK
testing: inspect.pyc OK
testing: linecache.pyc OK
testing: locale.pyc OK
testing: macfs.pyc OK
testing: macostools.pyc OK
testing: macpath.pyc OK
testing: macresource.pyc OK
testing: math.pyc OK
testing: native/__init__.pyc OK
testing: native/usb/__init__.pyc OK
testing: native/usb/_libusb.pyc OK
testing: native/usb/libusb.pyc OK
testing: native/usb/usb.pyc OK
testing: ntpath.pyc OK
testing: opcode.pyc OK
testing: operator.pyc OK
testing: os.pyc OK
testing: os2emxpath.pyc OK
testing: p_brew.pyc OK
testing: p_lg.pyc OK
testing: p_lgtm520.pyc OK
testing: p_lgvx4400.pyc OK
testing: p_lgvx6000.pyc OK
testing: p_sanyo.pyc OK
testing: parser.pyc OK
testing: pdb.pyc OK
testing: phonebook.pyc OK
testing: phonebookentryeditor.pyc OK
testing: popen2.pyc OK
testing: posixpath.pyc OK
testing: pprint.pyc OK
testing: profile.pyc OK
testing: prototypes.pyc OK
testing: pstats.pyc OK
testing: pubsub.pyc OK
testing: pwd.pyc OK
testing: py_compile.pyc OK
testing: random.pyc OK
testing: re.pyc OK
testing: repr.pyc OK
testing: select.pyc OK
testing: serial/__init__.pyc OK
testing: serial/serialjava.pyc OK
testing: serial/serialposix.pyc OK
testing: serial/serialutil.pyc OK
testing: serial/serialwin32.pyc OK
testing: sha.pyc OK
testing: socket.pyc OK
testing: sre.pyc OK
testing: sre_compile.pyc OK
testing: sre_constants.pyc OK
testing: sre_parse.pyc OK
testing: stat.pyc OK
testing: string.pyc OK
testing: strop.pyc OK
testing: struct.pyc OK
testing: symbol.pyc OK
testing: tempfile.pyc OK
testing: termios.pyc OK
testing: threading.pyc OK
testing: time.pyc OK
testing: token.pyc OK
testing: tokenize.pyc OK
testing: traceback.pyc OK
testing: types.pyc OK
testing: unittest.pyc OK
testing: usbscan.pyc OK
testing: version.pyc OK
testing: wallpaper.pyc OK
testing: warnings.pyc OK
testing: weakref.pyc OK
testing: webbrowser.pyc OK
testing: wx/__init__.pyc OK
testing: wx/grid.pyc OK
testing: wx/html.pyc OK
testing: wxPython/__init__.pyc OK
testing: wxPython/__version__.pyc OK
testing: wxPython/calendar.pyc OK
testing: wxPython/calendarc.pyc OK
testing: wxPython/clip_dnd.pyc OK
testing: wxPython/cmndlgs.pyc OK
testing: wxPython/controls.pyc OK
testing: wxPython/controls2.pyc OK
testing: wxPython/events.pyc OK
testing: wxPython/filesys.pyc OK
testing: wxPython/fonts.pyc OK
testing: wxPython/frames.pyc OK
testing: wxPython/gdi.pyc OK
testing: wxPython/gizmos.pyc OK
testing: wxPython/gizmosc.pyc OK
testing: wxPython/grid.pyc OK
testing: wxPython/gridc.pyc OK
testing: wxPython/help.pyc OK
testing: wxPython/helpc.pyc OK
testing: wxPython/html.pyc OK
testing: wxPython/htmlc.pyc OK
testing: wxPython/htmlhelp.pyc OK
testing: wxPython/image.pyc OK
testing: wxPython/lib/__init__.pyc OK
testing: wxPython/lib/colourdb.pyc OK
testing: wxPython/lib/grids.pyc OK
testing: wxPython/lib/intctrl.pyc OK
testing: wxPython/lib/maskededit.pyc OK
testing: wxPython/lib/mixins/__init__.pyc OK
testing: wxPython/lib/mixins/listctrl.pyc OK
testing: wxPython/lib/pubsub.pyc OK
testing: wxPython/lib/rcsizer.pyc OK
testing: wxPython/mdi.pyc OK
testing: wxPython/misc.pyc OK
testing: wxPython/misc2.pyc OK
testing: wxPython/printfw.pyc OK
testing: wxPython/sizers.pyc OK
testing: wxPython/stattool.pyc OK
testing: wxPython/streams.pyc OK
testing: wxPython/tools/__init__.pyc OK
testing: wxPython/tools/dbg.pyc OK
testing: wxPython/utils.pyc OK
testing: wxPython/windows.pyc OK
testing: wxPython/windows2.pyc OK
testing: wxPython/windows3.pyc OK
testing: wxPython/wx.pyc OK
testing: wxPython/wxc.pyc OK
testing: xyaptu.pyc OK
testing: zipfile.pyc OK
testing: zlib.pyc OK
Post by Roger Binns
Post by Steven Palm
14M dist
What size does the final distributable work out as. IIRC you
make a dimg (disk image) which is a compressed loopback filesystem.
14,233,205 bytes.

Yes, a disk image is one option for distribution. Either that or a
"Stuffit" file. Manyh of the items in this bundle are already
compressed, but when making a compressed disk image for distribution it
cuts it down to 5.8MB (6,113,771 bytes)
Post by Roger Binns
BTW I assume that sys.platform returns 'darwin'.
Yes, sorry 'bout that.
Post by Roger Binns
Also how do you encode the version numbers of BitPim itself.
One thing very different here from your Windows or Linux examples...
This is not an installer, this is a self-contained do-everything
application. The user would download this as either a disk image or
stuffit archive, and the resulting item can simply be drag-n-dropped
anywhere in the filesystem and run. It is entirely self-contained,
relying on no other system python parts/etc. Therefore, it's no trouble
to have ten different versions on the machine as long as the
application bundle has a different name or is in a different directory.
Very nice.
Post by Roger Binns
I have integrated the code into makedist.py. Thanks for your
continuing contributions on this.
THANK YOU. You have done a great work in bitpim, I'm just glad to do a
feeble little part. It makes me feel good to give even a little bit of
time back to the project.

-. ----. -.-- - -.--
Steve Palm - ***@n9yty.com
-. ----. -.-- - -.--
Roger Binns
2003-12-12 08:04:23 UTC
Permalink
# myapp.includePackages.append(c)

That probably doesn't work as c includes the .py in the filename.
Have you tried:

myapp.includePackages.append(c[:-3])

Roger

Loading...