Discussion:
[Bitpim-devel] Startup script for BitPim on Macs
Tom Pollard
2004-03-15 14:14:48 UTC
Permalink
Personally, I think that the startup script for BitPim should be a
shell script, rather than a Python script.
You should discuss this on bitpim-devel. On the other platforms, and
I had believed Mac as well, the binary distribution of BitPim includes
the Python interpretter and all modules needed (both ones written in
Python and the necessary shared libraries).
I'm not sure how things are done on the other platforms, but I would
have expected you to use the 'freeze' system for packaging a script as
a standalone executable, similarly to 'perl2exe' and it's ilk. These
systems package up the scripts, modules and dlls needed to run a script
in a zip archive with a binary loader prepended. The loader is
responsible for unpacking things, setting up the environment and
executing your script using the bundled interpreter. This lets you run
your script even if perl or python isn't installed at all on the target
machine.

The way BitPim's packaged for the Mac is simpler; MacOS X makes it easy
to do something along the lines of perl2exe without actually using a
single zip archive with a binary loader. But, you still need something
like a loader to kick things off. For BitPim, that's a small Python
script. This seems a little inelegant; since, this first script is the
guy who sets things up so that subsequent python scripts can use the
bundled python interpreter and modules, and so it requires the user to
Steven Palm
2004-03-15 15:59:14 UTC
Permalink
Post by Tom Pollard
I'm not sure how things are done on the other platforms, but I would
have expected you to use the 'freeze' system for packaging a script
as a standalone executable, similarly to 'perl2exe' and it's ilk.
Except, as far as I know, that does not work properly on MacOS X.
Prove me wrong, I won't mind. ;^)
Post by Tom Pollard
the user to have a separate, working Python installation.
Tom Pollard
2004-03-15 16:41:33 UTC
Permalink
Post by Steven Palm
Post by Tom Pollard
I'm not sure how things are done on the other platforms, but I would
have expected you to use the 'freeze' system for packaging a script
as a standalone executable, similarly to 'perl2exe' and it's ilk.
Except, as far as I know, that does not work properly on MacOS X.
Prove me wrong, I won't mind. ;^
I have no idea what the state of 'freeze' on Macs is, to be honest -
I'm a python newbie. (I do use ActiveState's 'PerlApp' under Windows
and the open-source 'PAR' package under Unix (including MacOS X) for
packaging Perl scripts as executables.)
Post by Steven Palm
Post by Tom Pollard
the user to have a separate, working Python installation.
Roger Binns
2004-03-15 17:47:38 UTC
Permalink
Post by Tom Pollard
I have no idea what the state of 'freeze' on Macs is, to be honest -
I'm a python newbie. (I do use ActiveState's 'PerlApp' under Windows
and the open-source 'PAR' package under Unix (including MacOS X) for
packaging Perl scripts as executables.)
For BitPim we do use different tools on each platform, as they all have
different strengths and weaknesses. It is definitely a goal that there
should be no need for anything to be already installed by the user.
That goal is met 100% on Windows and Linux.

On Windows Python itself is a DLL. py2exe produces a zip of all your
.py code, and then has a stub executable that loads the DLL, and sets
the zip as the code path. (See python-dev group for some proposals
for Python automatically finding the zip, removing the need for even
the stub executable).

Since zip files as an import source were only introduced in Python 2.3,
that approach doesn't work with earlier versions. On Linux we use
Python 2.2, and the cx_Freeze tool. That packages up the Python binary
with the code appended to the file and stub code to set things up
right (ie similar to zip but using custom code and archive format)

I believe that all the mechanisms will move over to the zip method
but that will set the minimum supported Python version as 2.3.

We also make sure we use "standard" stuff, and don't modify any other
components. For example I could install Python 2.3 on Linux and
base everything off that, but that would be a non-standard installation.

Similarly we could benefit from moving up to the developer version
of wxPython+wxWidgets. As Steven can tell you, I don't do that either :-)

If in your opinion something can be done better, then please contribute
that to the necessary place (in this case whoever makes bundlebuilder).
Also be very aware that there may be good reasons why they did things
the way they did. Also be aware that they may be concerned about backwards
compatibility (eg working with both Jaguar and Panther). But this is
what open source is about - you get the best from everyone rather than
one what secretive arbitrary team of people make.

Roger
Steven Palm
2004-03-15 18:38:01 UTC
Permalink
Post by Tom Pollard
the user to have a separate, working Python installation.
Steven Palm
2004-03-15 18:59:55 UTC
Permalink
I didn't realize that. I'd also be disinclined to muck with
bundlebuilder.py unless there was some stronger reason than this.
Maybe if the issue comes up again I'll lobby the bundlebuilder
maintainer to accept this change. Just on technical grounds, it seems
clearly the right way to go. Anyway, thanks for being willing to
discuss it.
Just an aside.... I was looking through bundlebuilder.py just now to
see if this could be remedied easily enough, and it appears that it
might be do-able... But you have to love this bit of code
(particularly the comment):

if self.standalone or self.semi_standalone:
# XXX we're screwed when the end user has deleted
# /usr/bin/python
hashbang = "/usr/bin/python"

... yes we are. :-)
Roger Binns
2004-03-15 20:02:36 UTC
Permalink
Post by Steven Palm
# XXX we're screwed when the end user has deleted
# /usr/bin/python
hashbang = "/usr/bin/python"
Gives new meaning to "standalone". Maybe they need a
"reallystandaloneireallymeanitandnooptionaldependenciesthistimeplease" :-)
(I have been known to name variables/functions like that sometimes when
experiencing a lot of frustration with other components).

Roger
Tom Pollard
2004-03-15 20:13:32 UTC
Permalink
Post by Roger Binns
Post by Steven Palm
# XXX we're screwed when the end user has deleted
# /usr/bin/python
hashbang = "/usr/bin/python"
Gives new meaning to "standalone". Maybe they need a
"reallystandaloneireallymeanitandnooptionaldependenciesthistimeplease" :-)
(I have been known to name variables/functions like that sometimes when
experiencing a lot of frustration with other components).
Yes, I noticed. MakeTheDamnThingRedraw() put a smile on my face when I
ran across it. It made me remember how glad I am not to be doing
X11/Motif programming anymore...

Tom

Michael Casteel
2004-03-15 17:34:16 UTC
Permalink
An optional install, yes, but one that is marked by default on every
installation. So, unless a user intentionally goes out of their way
when doing the install or upgrade to make it NOT install, it will be
there.
If it's optional, it's optional. Running ordinary apps under MacOS X
doesn't require its presence. My understanding is that it's mainly
there to support Terminal users and Unix geeks. If it was my
company's product, I'd insist on taking this simple step to make sure
my app ran on a minimal install of the OS.
Ah, if only more 'software professionals' were this professional about
their work! Back when I had a software company, I could have used as
many of you as I could get!
I didn't realize that. I'd also be disinclined to muck with
bundlebuilder.py unless there was some stronger reason than this.
Maybe if the issue comes up again I'll lobby the bundlebuilder
maintainer to accept this change. Just on technical grounds, it seems
Dang, where were you when I was hiring? Probably still in grade school
;-)
--
Mike Casteel
***@casteel.org Seattle, WA
Steven Palm
2004-03-15 19:16:43 UTC
Permalink
In any case, I've attached a shell script that could be used to
replace the current BitPim python startup script. It uses a small
shell function to extract the parent directory name from a filepath
because the 'dirname' shell utility (like Python) is only there in
MacOS X if the user installs the optional BSD package. Otherwise, it's
just about line-for-line equivalent with the python version.
I'm not interested in removing the BSD subsystem from my computer to
test, but I see you state that `dirname` is not available unless you
install the BSD subsystem. You are using `sed`, however, so that one
*is* installed? Given this kind of thinking, it's probably somewhat
arbitrary what commands Apple puts on your system as a CORE package
versus the BSD package, because I would land `sed` as a part of the BSD
subsystem as well, but it apparently is not. This makes it hard to be
foolproof for future OS revisions as well if you write a shell script
that depends on certain commands being there (albeit minimal in this
case.
Tom Pollard
2004-03-15 19:40:41 UTC
Permalink
Post by Steven Palm
In any case, I've attached a shell script that could be used to
replace the current BitPim python startup script. It uses a small
shell function to extract the parent directory name from a filepath
because the 'dirname' shell utility (like Python) is only there in
MacOS X if the user installs the optional BSD package. Otherwise,
it's just about line-for-line equivalent with the python version.
I'm not interested in removing the BSD subsystem from my computer to
test, but I see you state that `dirname` is not available unless you
install the BSD subsystem. You are using `sed`, however, so that one
*is* installed?
Yes, sed and awk are always installed on a minimal unix system.
Post by Steven Palm
Given this kind of thinking, it's probably somewhat arbitrary what
commands Apple puts on your system as a CORE package versus the BSD
package, because I would land `sed` as a part of the BSD subsystem as
well, but it apparently is not.
It's understandable that this looks arbitrary to you, but it's not,
really. I was surprised that there was a 'dirname' utility at all, as
soon as I saw it, because I know it's not standard. That prompted me
to check the Receipts directory to find which package it was installed
with, and I found it was part of the BSD install.

(I have a perl script that lets you search your Receipts to see who
installed what, if that sounds useful. The Linux rpm system lets you
ask questions like that easily, but I couldn't find anything for MacOS
X that was comparable. My perl script isn't really suitable for
release, IMO, because it's kind of slow - it has to reread all of the
receipts each time you run the script and that's an annoying delay. It
ought to cache that stuff somewhere, since it only rarely changes.)
Post by Steven Palm
This makes it hard to be foolproof for future OS revisions as well if
you write a shell script that depends on certain commands being there
(albeit minimal in this case.
Not really. I've been dealing with cross-platform Unix issues for most
of my career, and know how to write a portable Bourne shell script that
you can expect to run pretty much anywhere. (Not a terribly vital job
skill nowadays, unfortunately.) Sed and awk are always safe. The main
challenge is recognizing and avoiding shell features that are bash- or
ksh-specific. You might be surprised how much you can still do within
these limitations.


Cheers,

Tom
Roger Binns
2004-03-15 20:00:44 UTC
Permalink
Post by Tom Pollard
Not really. I've been dealing with cross-platform Unix issues for most
of my career, and know how to write a portable Bourne shell script that
you can expect to run pretty much anywhere.
Me too! I used to work on machines that had multiple universes (as they
called it). You could have a sysv environment or a bsd environment
with commands scattered between them, and lots of other fun issues.
Post by Tom Pollard
Sed and awk are always safe.
They may always exist, but they don't necessarily always run with
non-trivial data. Things like arbitrary limits on line length (such
as 512 or 1024 chars), breaking on any files not having UNIX end of
lines, limits on the number of variables etc.
Post by Tom Pollard
The main
challenge is recognizing and avoiding shell features that are bash- or
ksh-specific. You might be surprised how much you can still do within
these limitations.
Heck, I still don't use shell functions except when I occassionally
remember they exist :-) I ended up writing a macro system in TCL
that had a library of shell code, and then expanded it for the
main shell scripts. It turned out to be very useful as there is
so much stuff that doesn't work between machines, even modern
ones (for example try killing one of your own processes named foo,
or finding out how much free disk space there is for a proposed
directory name).
Post by Tom Pollard
(Not a terribly vital job skill nowadays, unfortunately.)
It isn't the skill that matters, but more the way of thinking that
does. I am glad that a lot of these lower level issues are non-issues
any more, while the thinking is still very relevant. For example here
is another great programmer complaining about the very same issue:

http://www.joelonsoftware.com/articles/PleaseLinker.html

Roger
Loading...