Mercurial > hg > carton

annotate carton.py @ 26:e6ee0410ceef

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix bug where os.path.sep is wrongly interpreted
author Jeff Hammel <jhammel@mozilla.com>
date Sun, 10 Jul 2011 18:37:42 -0700
parents bb19d3dac4c5
children dace84448c25
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
1 #!/usr/bin/env python
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
2
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
3 """
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
4 make a self-extracting virtualenv from directories or URLs
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
5 of packages
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
6
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
7 To package up all files in a virtualenvs source directory (e.g.):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
8
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
9 python path/to/carton.py mozmill mozmill/src/*
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
10
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
11 This will create a self-extracting file, `mozmill.py`, that will unfold
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
12 a virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
13 """
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
14
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
15 # imports
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
16 import os
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
17 import sys
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
18 import subprocess
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
19 import tarfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
20 import tempfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
21 import urllib2
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
22 from optparse import OptionParser
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
23 from StringIO import StringIO
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
24
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
25 # global variables
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
26 usage = "%prog [options] environment_name directory|url [...]"
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
27 virtualenv_url = 'http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.6.1.tar.gz'
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
28 template = """#!/usr/bin/env python
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
29
15
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
30 "create a virtualenv at %(ENV)s"
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
31
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
32 import os
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
33 import shutil
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
34 import subprocess
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
35 import sys
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
36 import tarfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
37 import tempfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
38 from optparse import OptionParser
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
39 from StringIO import StringIO
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
40
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
41 try:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
42 call = subprocess.check_call
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
43 except AttributeError:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
44 # old python; boo :(
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
45 call = subprocess.call
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
46
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
47 # virtualenv name
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
48 ENV='''%(ENV)s'''
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
49
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
50 # packed files
3
75919adb199a use compression, but it doesnt seem to help much
Jeff Hammel <jhammel@mozilla.com>
parents: 0
diff changeset
51 VIRTUAL_ENV='''%(VIRTUAL_ENV)s'''.decode('base64').decode('zlib')
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
52 PACKAGE_SOURCES=%(PACKAGE_SOURCES)s
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
53 CARTON=%(CARTON)s
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
54
15
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
55 # parse options
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
56 usage = os.path.basename(sys.argv[0]) + ' [options]'
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
57 parser = OptionParser(usage=usage, description=__doc__)
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
58 parser.add_option('--env', dest='env', help="environment name [DEFAULT: " + ENV + "]")
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
59 options, args = parser.parse_args()
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
60 if options.env:
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
61 ENV = options.env
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
62
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
63 # unpack virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
64 tempdir = tempfile.mkdtemp()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
65 buffer = StringIO()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
66 buffer.write(VIRTUAL_ENV)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
67 buffer.seek(0)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
68 tf = tarfile.open(mode='r', fileobj=buffer)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
69 tf.extractall(tempdir)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
70
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
71 # find the virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
72 for root, dirs, files in os.walk(tempdir):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
73 if 'virtualenv.py' in files:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
74 virtualenv = os.path.join(root, 'virtualenv.py')
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
75 break
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
76 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
77 raise Exception("virtualenv.py not found in " + tempdir)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
78
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
79 # create the virtualenv
17
b05f5f1ec26e pop PYTHONHOME as this will fuxor things
Jeff Hammel <jhammel@mozilla.com>
parents: 16
diff changeset
80 os.environ.pop('PYTHONHOME', None)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
81 call([sys.executable, virtualenv, ENV])
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
82
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
83 # find the bin/scripts directory
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
84 for i in ('bin', 'Scripts'):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
85 scripts_dir = os.path.abspath(os.path.join(ENV, i))
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
86 if os.path.exists(scripts_dir):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
87 break
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
88 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
89 raise Exception("Scripts directory not found in " + ENV)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
90
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
91 # find the virtualenv's python
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
92 for i in ('python', 'python.exe'):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
93 python = os.path.join(scripts_dir, i)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
94 if os.path.exists(python):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
95 break
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
96 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
97 raise Exception("python not found in " + scripts_dir)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
98
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
99 # unpack the sources and setup for development
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
100 srcdir = os.path.join(ENV, 'src')
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
101 os.mkdir(srcdir)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
102 setup_pys = set()
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
103 for source in PACKAGE_SOURCES:
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
104 source = source.decode('base64').decode('zlib')
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
105 buffer = StringIO()
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
106 buffer.write(source)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
107 buffer.seek(0)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
108 tf = tarfile.open(mode='r', fileobj=buffer)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
109 tf.extractall(srcdir)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
110
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
111 # setup sources for development if there are any new setup.py files
16
c91abbdc871b * add TODO: dependencies
Jeff Hammel <jhammel@mozilla.com>
parents: 15
diff changeset
112 # TODO: ideally this would figure out dependency order for you
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
113 for i in os.listdir(srcdir):
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
114 if i in setup_pys:
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
115 continue
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
116 subdir = os.path.join(srcdir, i)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
117 if os.path.exists(os.path.join(srcdir, i, 'setup.py')):
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
118 call([python, 'setup.py', 'develop'], cwd=subdir)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
119 setup_pys.add(i)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
120
22
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
121 # add virtualenv to the virtualenv (!)
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
122 virtualenv_dir = os.path.dirname(virtualenv)
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
123 if os.path.exists(os.path.join(virtualenv_dir, 'setup.py')):
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
124 call([python, 'setup.py', 'install'], cwd=virtualenv_dir, stdout=subprocess.PIPE)
22
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
125
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
126 # TODO:
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
127
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
128 # - add carton to the virtualenv (!)
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
129 # if CARTON:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
130 # CARTON = CARTON.decode('base64').decode('zlib')
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
131 # carton_filename = os.path.join(scripts_dir, 'carton.py')
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
132 # f = file(carton_filename, 'w')
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
133 # f.write(CARTON)
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
134 # f.close()
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
135 # try:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
136 # os.chmod(carton_filename, 0755)
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
137 # except:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
138 # # you probably don't have os.chmod
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
139 # pass
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
140
21
46882eaebb59 consolidate and order TODO for generated script
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
141 # - cleanup tempdir
46882eaebb59 consolidate and order TODO for generated script
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
142 # shutil.rmtree(tempdir)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
143 """
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
144
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
145 def isURL(path):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
146 return path.startswith('http://') or path.startswith('https://')
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
147
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
148 try:
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
149 call = subprocess.check_call
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
150 except AttributeError:
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
151 # old python; boo :(
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
152 call = subprocess.call
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
153
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
154 def main(args=sys.argv[1:]):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
155
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
156 # parse CLI arguments
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
157 parser = OptionParser(usage=usage, description=__doc__)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
158 parser.add_option('-o', dest='outfile',
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
159 help="specify outfile; otherwise it will come from environment_name")
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
160 parser.add_option('-p', '--package', dest='package',
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
161 action='store_true', default=False,
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
162 help="create python packages from sources; do not take entire subdirectory")
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
163 parser.add_option('--virtualenv', dest='virtualenv',
12
542b46ac4e28 fix description more better
Jeff Hammel <jhammel@mozilla.com>
parents: 11
diff changeset
164 help="use this virtualenv URL or file tarball")
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
165 options, args = parser.parse_args(args)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
166 if len(args) < 2:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
167 parser.print_usage()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
168 parser.exit()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
169 environment = args[0]
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
170 if environment.endswith('.py'):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
171 # stop on .py; will add it in later
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
172 environment = environment[:-3]
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
173 sources = args[1:]
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
174
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
175 # tar up the sources
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
176 source_array = []
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
177 for source in sources:
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
178 buffer = None
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
179
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
180 if isURL(source):
14
c474362cf69a allow fetching of remote tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 13
diff changeset
181 # remote tarball or resource
c474362cf69a allow fetching of remote tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 13
diff changeset
182 buffer = urllib2.urlopen(source).read()
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
183 else:
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
184 # local directory or tarball
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
185 assert os.path.exists(source), "%s does not exist" % source
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
186
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
187 # package up the source if applicable
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
188 if options.package and os.path.exists(os.path.join(source, 'setup.py')):
24
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
189
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
190 # create a .tar.gz package
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
191 call([sys.executable, 'setup.py', 'sdist'], cwd=source, stdout=subprocess.PIPE)
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
192 dist_dir = os.path.join(source, 'dist')
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
193 assert os.path.isdir(dist_dir), "dist directory not created in %s" % source
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
194 tarfiles = [i for i in os.listdir(dist_dir)
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
195 if i.endswith('.tar.gz')]
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
196 assert tarfiles, "no .tar.gz files found in %s" % dist_dir
24
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
197
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
198 # use the last modified tarball
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
199 def last_modified(filename):
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
200 return os.path.getmtime(os.path.join(dist_dir, filename))
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
201 tarfiles.sort(key=last_modified)
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
202 source = os.path.join(dist_dir, tarfiles[-1])
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
203
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
204 if (not os.path.isdir(source)) and tarfile.is_tarfile(source):
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
205 # check for a tarball
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
206 buffer = file(source).read()
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
207 else:
24
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
208 # add other sources (files and directories) to the archive
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
209 source_buffer = StringIO()
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
210 source_tar = tarfile.open(mode="w:gz", fileobj=source_buffer)
26
e6ee0410ceef fix bug where os.path.sep is wrongly interpreted
Jeff Hammel <jhammel@mozilla.com>
parents: 24
diff changeset
211 source_tar.add(source, arcname=os.path.basename(source.rstrip(os.path.sep)))
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
212 source_tar.close()
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
213 buffer = source_buffer.getvalue()
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
214
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
215 # could use git, hg, etc repos. but probably shouldn't
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
216
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
217 source_array.append(buffer.encode('zlib').encode('base64'))
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
218
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
219 # tar up virtualenv if not available
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
220 if options.virtualenv:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
221 if isURL(options.virtualenv):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
222 globals()['VIRTUAL_ENV'] = urllib2.urlopen(options.virtualenv).read()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
223 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
224 assert os.path.exists(options.virtualenv)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
225 if os.path.isdir(options.virtualenv):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
226 raise NotImplementedError("Hypothetically you should be able to use a local directory or tarball, but I haven't done this yet")
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
227 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
228 # assert a tarfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
229 assert tarfile.is_tarfile(options.virtualenv), "%s must be a tar file" % options.virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
230 globals()['VIRTUAL_ENV'] = file(options.virtualenv).read()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
231 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
232 globals()['VIRTUAL_ENV'] = urllib2.urlopen(virtualenv_url).read()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
233 # TODO: used the below hashed value of VIRTUAL_ENV if set
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
234 # (set that with another file)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
235
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
236 # get the contents of this file
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
237 carton = None
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
238 try:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
239 if __file__:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
240 filename = __file__.rstrip('c') # avoid pyfiles
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
241 if os.path.exists(filename):
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
242 carton = file(filename).read().encode('zlib').encode('base64')
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
243 except NameError:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
244 pass
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
245
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
246 # interpolate "template" -> output
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
247 outfile = options.outfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
248 if outfile is None:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
249 outfile = environment + '.py'
3
75919adb199a use compression, but it doesnt seem to help much
Jeff Hammel <jhammel@mozilla.com>
parents: 0
diff changeset
250 variables = {'VIRTUAL_ENV': VIRTUAL_ENV.encode('zlib').encode('base64'),
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
251 'ENV': environment,
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
252 'CARTON': repr(carton),
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
253 'PACKAGE_SOURCES': repr(source_array)}
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
254 f = file(outfile, 'w')
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
255 f.write(template % variables)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
256 f.close()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
257 try:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
258 os.chmod(outfile, 0755)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
259 except:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
260 # you probably don't have os.chmod
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
261 pass
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
262
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
263 VIRTUAL_ENV = """"""
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
264
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
265 if __name__ == '__main__':
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
266 main()