jvmバックエンドに完敗 PyPy Advent Calendar 26日 + 1日目(27日目?)

まさかのPyPy Advent Calendar 2011 26日 + 1日目(27日目?)。

この記事はaodagさんのpypy cliバックエンドに完敗 aodag's posterousインスパイヤされて書かれてます。

結論書くとjvmバックエンドに完敗:(

追記

この記事での敗北はid:shomah4aさんが勝利を収めました。
下記がそのまとめ記事です。

事前準備

以前、PyPyのRPythonToolChainで実装したOok!を使います。

hg clone https://bitbucket.org/rokujyouhitoma/lang-ook

また、PyPyのリポジトリに入ってるtranslate.pyを使うのでPyPyのリポジトリをcloneする必要があります。

$ hg clone https://bitbucket.org/pypy/pypy

環境

  • Mac OS X Lion 10.7.2
  • 1.7 GHz Intel Core i5
  • 4 GB 1333 MHz DDR3

ビルド

(_)ike-no-MacBook-Air% python ~/workspace/pypy/pypy/translator/goal/translate.py --backend=jvm ook.py
[platform:msg] Setting platform to 'host' cc=None
[translation:info] Translating target as defined by ook
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/gcctest.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/gcctest.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/gcctest.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/gcctest
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_0.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_0.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_0.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_0
[translation] translate.py configuration:
[translation] [translate]
[translation]     targetspec = ook
[translation] translation configuration:
[translation] [translation]
[translation]     backend = jvm
[translation]     [backendopt]
[translation]         clever_malloc_removal = False
[translation]         constfold = False
[translation]     continuation = False
[translation]     gc = boehm
[translation]     gcremovetypeptr = False
[translation]     gcrootfinder = n/a
[translation]     gctransformer = boehm
[translation]     list_comprehension_operations = True
[translation]     type_system = ootype
[translation]     withsmallfuncsets = 5
[translation:info] Annotating&simplifying...
[translation:info] with policy: pypy.annotation.policy.AnnotatorPolicy
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_1.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_1.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_1.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_1
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_2.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_2.o
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_3.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_3.o
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_4.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_4.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_4.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_4
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_5.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_5.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_5.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_5
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_6.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_6.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_6.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_6
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_7.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_7.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_7.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_7
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_8.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_8.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_8.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_8
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_9.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_9.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_9.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_9
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_10.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_10.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_10.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_10
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_11.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_11.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_11.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_11
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_12.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_12.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_12.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_12
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_13.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_13.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_13.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_13
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_14.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_14.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_14.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_14
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_15.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_15.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_15.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_15
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_16.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_16.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_16.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_16
[platform:execute] gcc-4.2 -c -arch x86_64 -O3 -fomit-frame-pointer -mdynamic-no-pic /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_17.c -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_17.o
[platform:execute] gcc-4.2 /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_17.o -arch x86_64 -o /var/folders/lc/vw5blb150gxd3_xwxj2k3s080000gn/T/usession-default-0/platcheck_17
...+++++++++++
[translation:info] -- someobjectness  0% (0 of 14 functions polluted by SomeObjects)
[translation:info] ootyping...
...+++++++++++++++*************************%
[rtyper] specializing: 100 / 204 blocks   (49%)
[rtyper] specializing: 200 / 204 blocks   (98%)
[rtyper] -=- specialized 204 blocks -=-
...
[rtyper] -=- specialized 8 more blocks -=-
[translation:info] ootype back-end optimisations...
[backendopt:inlining] phase with threshold factor: 32.4
[backendopt:inlining] heuristic: pypy.translator.backendopt.inline.inlining_heuristic
...+++++++++++++++*************************%
[backendopt:inlining] inlined 75 callsites.
[backendopt:malloc] starting malloc removal
...+
[backendopt:malloc] removed 6 simple mallocs in total
[backendopt:mergeifblocks] starting to merge if blocks
[translation:info] Generating JVM source...
...++++++++
[rtyper] -=- specialized 22 more blocks -=-
[translation:info] Wrote JVM code
[translation:info] Compiling JVM source...
[jvm] Compiling java classes
[translation:info] Compiled JVM source
[translation:info] Creating jar file
[Timer] Timings:
[Timer] annotate                       ---  2.7 s
[Timer] rtype_ootype                   ---  2.2 s
[Timer] backendopt_ootype              ---  1.1 s
[Timer] source_jvm                     ---  0.7 s
[Timer] compile_jvm                    ---  4.5 s
[Timer] =========================================
[Timer] Total:                         --- 11.2 s

ビルドは順調に10秒ちょいで終わり。
ook-jvmとook-jvm.jarができあがり。


ook-jvm

ook-jvmはバイナリかな?とのぞいてみると、シェルスクリプトでした。

(_)ike-no-MacBook-Air% cat ook-jvm
#!/bin/bash
LEDIT=`type -p ledit`
EXE=`readlink $0`
if [ -z $EXE ]
then
    EXE=$0
fi
$LEDIT java -Xmx256m -jar $EXE.jar "$@"

実行

さて、ook-jvmを実行してみる。

(_)ike-no-MacBook-Air% ./ook-jvm
Exception in thread "main" java.lang.VerifyError: (class: pypy/ConstantInit_0, method: constant_init signature: ()V) Expecting to find long on stack
	at pypy.ConstantInit.init(ConstantInit.j:6)
	at pypy.Main.<clinit>(Main.j:26)

java.lang.VerifyErrorが発生...。

jarを直接実行

うむむ。jarをjavaコマンドで直接叩いてみよう。

(_)ike-no-MacBook-Air% java -jar ./ook-jvm.jar
Exception in thread "main" java.lang.VerifyError: (class: pypy/ConstantInit_0, method: constant_init signature: ()V) Expecting to find long on stack
	at pypy.ConstantInit.init(ConstantInit.j:6)
	at pypy.Main.<clinit>(Main.j:26)


うーむ...。"java.lang.VerifyError"をGoogle先生で聞いたら、pypy-devのMLにたどり着く。

I suppose it's a 32/64 bit issue again: the problem is that when you run
./translate.py with a 64 bit python, it assumes that 'int' variables are 64
bits, but on the JVM they are always 32 bit. For what I know, the CLI backend
has exactly the same problem.

http://mail.python.org/pipermail/pypy-dev/2009-December/005593.html

32bitじゃないといけないの...( ̄д ̄)エー


>プロセッサ名: Intel Core i5

だから...マイマシンは64ビットで間違いない。

プロセッサ名 32 ビット/64 ビット
Intel Core Solo 32 ビット
Intel Core Duo 32 ビット
Intel Core 2 Duo 64 ビット
Intel Quad-Core Xeon 64 ビット
Dual-Core Intel Xeon 64 ビット
Quad-Core Intel Xeon 64 ビット
Core i3 64 ビット
Core i5 64 ビット
Core i7 64 ビット

http://support.apple.com/kb/HT3696?viewlocale=ja_JP&locale=ja_JP

32bit環境無いので機会があれば試す。
CLIに似たような問題があるって書いてある...φ(..)メモメモ


追記


下記のご指摘頂きました!

  • class file レベルで 32 bit とか関係無いような…… (匿名ペンギンさん)
  • あの引用されてる英文読むと, "64bit" は Python にかかってます(匿名ペンギンさん)
  • そもそも JVM 上で 32/64 bit の差はほとんど無いはずです(匿名ペンギンさん)
  • 64bit python では int は 64bit(匿名ペンギンさん)