MemoryError when importing py5

[fuho]

Hi,
I am struggling to get py5 running on my laptop running EndeavourOS. I am using venv, installed java using install-jdk (Installed 17 as shown in the docs), py5 installed as py5[extra]. When importig py5 I get:

❯ ipython
Python 3.13.1 (main, Dec  4 2024, 18:05:56) [GCC 14.2.1 20240910]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.32.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import py5
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 import py5

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/py5/__init__.py:111
    108 except:
    109     started_jvm = False
--> 111 debug_info = py5_tools.get_jvm_debug_info()
    112 java_version = debug_info["jvm version"][0]
    113 if not started_jvm or java_version < 17:

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/py5_tools/jvm.py:169, in get_jvm_debug_info()
    167 out = dict()
    168 out["JAVA_HOME environment variable"] = os.environ.get("JAVA_HOME", "<not set>")
--> 169 out["jvm version"] = jpype.getJVMVersion()
    170 out["default jvm path"] = jpype.getDefaultJVMPath()
    171 return out

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_core.py:467, in getJVMVersion()
    464     return (0, 0, 0)
    466 import re
--> 467 runtime = _jpype.JClass('java.lang.Runtime')
    468 version = runtime.class_.getPackage().getImplementationVersion()
    470 # Java 9+ has a version method

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jclass.py:99, in JClass.__new__(cls, jc, loader, initialize)
     96     return ret
     98 # Pass to class factory to create the type
---> 99 return _jpype._getClass(jc)

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jclass.py:147, in _jclassPost(res, *args)
    143 # Attach public inner classes we find
    144 #   Due to bootstrapping, we must wait until java.lang.Class is defined
    145 #   before we can access the class structures.
    146 if _jpype._java_lang_Class:
--> 147     for cls in res.class_.getDeclaredClasses():
    148         if cls.getModifiers() & 1 == 0:
    149             continue

MemoryError: 

In [2]: 

I don't have JAVA_HOME set, java is installed in ~/.jdk/jdk-17...

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                              ░▒▓ 5m 57s  play-with-ai    22:48:56 
❯ which java
java not found

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                       ░▒▓ play-with-ai    22:48:59 
❯ ls -al ~/.jdk/jdk-17.0.14+7/
total 8
drwxr-xr-x 1 fuho fuho   86 Jan 22 00:41 .
drwxr-xr-x 1 fuho fuho   26 Feb 17 22:30 ..
drwxr-xr-x 1 fuho fuho  330 Jan 22 00:41 bin
drwxr-xr-x 1 fuho fuho  138 Jan 22 00:37 conf
drwxr-xr-x 1 fuho fuho  140 Jan 22 00:37 include
drwxr-xr-x 1 fuho fuho 2642 Jan 22 00:37 jmods
drwxr-xr-x 1 fuho fuho 1942 Jan 22 00:37 legal
drwxr-xr-x 1 fuho fuho 1168 Jan 22 00:41 lib
drwxr-xr-x 1 fuho fuho    8 Jan 22 00:37 man
-rw-r--r-- 1 fuho fuho 2400 Jan 22 00:41 NOTICE
-rw-r--r-- 1 fuho fuho 1626 Jan 22 00:41 release

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                       ░▒▓ play-with-ai    22:49:28 
❯ ~/.jdk/jdk-17.0.14+7/bin/java --version
openjdk 17.0.14 2025-01-21
OpenJDK Runtime Environment Temurin-17.0.14+7 (build 17.0.14+7)
OpenJDK 64-Bit Server VM Temurin-17.0.14+7 (build 17.0.14+7, mixed mode, sharing)

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                       ░▒▓ play-with-ai    22:49:48 
❯ echo $JAVA_HOME


 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                       ░▒▓ play-with-ai    22:50:25 

[hx2A]

Hi @fuho !

I'm not sure what is happening here, but let's figure it out.

First, can you tell me if you can start the JVM with JPype?

>> import jpype
>> jpype.startJVM()
>>

That MemoryError is useless; there is no text afterwards explaining what is wrong there.

My first thought was to try setting $JAVA_HOME to see if that helps, but your error message is not consistent with what I've previously seen for that kind of problem. I would try setting it anyway but I am not optimistic it will make a difference.

My guess is that there's some OS library that the JDK depends on that has not been installed on your machine. Maybe try installing openjdk through your EndeavourOS's package manager, which should install any dependent libraries also?

If neither of those help and you still get that error, I would suggest asking the JPype folks for guidance.

[fuho]

HEllo @hx2A ,
thanks for super fast response! I tried starting JVM as you said and got back messages about not being able to find JVM.

In [1]: import jpype

In [2]: jpype.startJVM()
---------------------------------------------------------------------------
JVMNotFoundException                      Traceback (most recent call last)
Cell In[2], line 1
----> 1 jpype.startJVM()

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_core.py:219, in startJVM(jvmpath, classpath, ignoreUnrecognized, convertStrings, interrupt, *jvmargs)
    216         jvmargs = jvmargs[1:]
    218 if not jvmpath:
--> 219     jvmpath = getDefaultJVMPath()
    220 else:
    221     # Allow the path to be a PathLike.
    222     jvmpath = os.fspath(jvmpath)

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jvmfinder.py:74, in getDefaultJVMPath()
     72 else:
     73     finder = LinuxJVMFinder()
---> 74 return finder.get_jvm_path()

File ~/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jvmfinder.py:212, in JVMFinder.get_jvm_path(self)
    210 if jvm_notsupport_ext is not None:
    211     raise jvm_notsupport_ext
--> 212 raise JVMNotFoundException("No JVM shared library file ({0}) "
    213                            "found. Try setting up the JAVA_HOME "
    214                            "environment variable properly."
    215                            .format(self._libfile))

JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.

I actually made sure there is no JAVA_HOME set on purpose and installed it using install-jdk, thinking that it would be found (as mentioned here).

[hx2A]

I actually made sure there is no JAVA_HOME set on purpose and installed it using install-jdk, thinking that it would be found (as mentioned here).

Right, but py5 knows to look for Java in the location where install-jdk installs Java. JPype doesn't know about that and won't know to look there.

Can you try installing Java though your OS? I think in your case, that will work better.

[fuho]

Ok, added JAVA_HOME to my .zshrc and sourced it (new terminal). jpype now sees the java path but MemoryError reappears.

>>> import jpype
>>> jpype.getDefaultJVMPath()
'/home/fuho/.jdk/jdk-17.0.14+7/lib/server/libjvm.so'
>>> jpype.startJVM()
Traceback (most recent call last):
  File "<python-input-9>", line 1, in <module>
    jpype.startJVM()
    ~~~~~~~~~~~~~~^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_core.py", line 250, in startJVM
    initializeResources()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_core.py", line 275, in initializeResources
    _jpype._java_lang_String = _jpype.JClass("java.lang.String")
                               ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jclass.py", line 99, in __new__
    return _jpype._getClass(jc)
           ~~~~~~~~~~~~~~~~^^^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jclass.py", line 147, in _jclassPost
    for cls in res.class_.getDeclaredClasses():
               ^^^^^^^^^^
MemoryError
>>> 

I'll remove the $HOME/.jdk folder then and install system wide java17 temurin.

[fuho]

Removed $HOME/.jdk folder and installed system-wide Java 17:

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                         ░▒▓ play-with-ai    00:38:20 
❯ echo $JAVA_HOME                      
/usr/bin/java

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                         ░▒▓ play-with-ai    00:38:23 
❯ java --version 
openjdk 17.0.14 2025-01-21
OpenJDK Runtime Environment Temurin-17.0.14+7 (build 17.0.14+7)
OpenJDK 64-Bit Server VM Temurin-17.0.14+7 (build 17.0.14+7, mixed mode, sharing)

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                         ░▒▓ play-with-ai    00:38:31 
❯ python -c "import jpype; jpype.startJVM()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import jpype; jpype.startJVM()
                  ~~~~~~~~~~~~~~^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_core.py", line 250, in startJVM
    initializeResources()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_core.py", line 275, in initializeResources
    _jpype._java_lang_String = _jpype.JClass("java.lang.String")
                               ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jclass.py", line 99, in __new__
    return _jpype._getClass(jc)
           ~~~~~~~~~~~~~~~~^^^^
  File "/home/fuho/PycharmProjects/play-with-ai/.venv/lib/python3.13/site-packages/jpype/_jclass.py", line 147, in _jclassPost
    for cls in res.class_.getDeclaredClasses():
               ^^^^^^^^^^
MemoryError

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                         ░▒▓ play-with-ai    00:38:38 
❯ python -c "import jpype; print(jpype.getDefaultJVMPath())"
/usr/lib/jvm/java-17-temurin/lib/server/libjvm.so

 ~/PycharmProjects/play-with-ai  main +3 !3 ?2 ▓▒░                                                                         ░▒▓ play-with-ai    00:38:55 
❯ 

[hx2A]

OK, you are still getting that enigmatic MemoryError. I don't know what is causing that. How much memory does your machine have?

I would suggest opening an issue with JPype and reference this issue. They might have seen this before.

[fuho]

64GB of RAM and it's not really claiming any...
I tried to debug it but got lost, I am sure I am jsut doing something stupid, jsut don't know what :D

[hx2A]

The JPype docs say for Ubuntu users, you need to install this:

sudo apt-get install g++ python3-dev

I know Endeavour is not a Debian flavor of Linux but perhaps there is something unique about it where it needs certain packages to be installed first?

[fuho]

Interesting, lemme try that...

[fuho]

Ok, checked the libs and found out I already have all of them installed.

[hx2A]

Ok, checked the libs and found out I already have all of them installed.

I'm out of ideas here. Try opening an issue with JPype and see what they say.

[fuho]

Yeah I think you're right...

[fuho]

@hx2A Actually just fixed this issue by force upgrading (against py5 requirements) jpype 1.5.0 (named JPype1) to 1.5.2 (named jpype1).

[hx2A]

@hx2A Actually just fixed this issue by force upgrading (against py5 requirements) jpype 1.5.0 (named JPype1) to 1.5.2 (named jpype1).

Oh, OK! Glad you got that working!

And yes I need to do a release to change the JPype requirement to >= 1.5.2. I made it == 1.5.0 because 1.5.1 had complicated problems for us.

I'm seriously swamped right now and haven't added much to py5 since the last release but maybe I should do a release anyway, just to make this change? I will see what I can do.

[villares]

@hx2A Actually just fixed this issue by force upgrading (against py5 requirements) jpype 1.5.0 (named JPype1) to 1.5.2 (named jpype1).

Thank you @fuho! I have to test JPype 1.5.2 on Windows to see if it fixed some other troubling issues, 1.5.1 broke stuff and caused us to pin it to 1.5.0 on py5. Sounds like progress!