Swith Backend to PyGhidra (Ghidra 12); Add Server/Client for DecompilerInterface

[mahaloz]

Tied to binsync/binsync#408, closes #123.

[mahaloz]

Current Progress

• Configure panel starts correctly and allows connection
• Syncing functions works (at least on fmt binary)
• Callbacks for function changes work (at least on fmt binary)
• Force push works!

Bugs

• Force-deleting the binsync lock on a project fails and does not open
• Closing the binsync panel does not delete the binsync.lock inside the git project
• None of the gui_* functions work, including tracking current address and jumps

TODO

• fix above bugs
• refactor how printing and logging works
• update example plugins to no longer have Ghidra bridge sutff

[mahaloz]

What is Stalling This PR?

Aside from the TODOs and bugs above in the thread, there is two major problems delaying the completion of this PR. Both are major.

Problem 1: Starting a QApplication in a JPype thread causes a segfault on MacOS

Running the following code in a Python3 script in Ghidra will segfault on Mac

from PySide6.QtWidgets import QApplication
QApplication()

End of the stack trace on MacOS:

...
50  libjvm.dylib                  	       0x10407678c JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*) + 600
51  libjvm.dylib                  	       0x104075b4c JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*) + 264
52  libjvm.dylib                  	       0x104075c14 JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*) + 100
53  libjvm.dylib                  	       0x10411b2b0 thread_entry(JavaThread*, JavaThread*) + 156
54  libjvm.dylib                  	       0x104086984 JavaThread::thread_main_inner() + 132
55  libjvm.dylib                  	       0x1040867cc JavaThread::run() + 336
56  libjvm.dylib                  	       0x10448d5f8 Thread::call_run() + 140
57  libjvm.dylib                  	       0x1043248ec thread_native_entry(Thread*) + 204
58  libsystem_pthread.dylib       	       0x18b243c0c _pthread_start + 136
59  libsystem_pthread.dylib       	       0x18b23eb80 thread_start + 8

I speculate this is due to the hard requirements of being on the main thread when doing any GUI in macOS.

I see only two possible solutions:

1. Find a hack around this, like a way to start a Python process on the main thread, move all work to a background thread, but keep Qt on top.
2. Abandon starting a Qt instance in the same PyGhidra process. Start a second process that has the GUI, and communicate with rpyc to the server in the main Ghidra process. (progress started in 567a4aa, but motivation lost)

This bug is critical and would drop all macOS support if merged without completion.

Problem 2: Subclassing Java Classes is Broken in JPype

We can't subclass any class from Ghidra. This is a problem for ProgramLocationContextAction, which we need.

The solution is to make a class that can be implemented that uses ProgramLocationContextAction (in Java), and ship that with our Python. I don't yet know how I'd make this Jar ship with BinSync so it's some work.

Want to Help?

If you know insane things about Qt on MacOS, I could really use your advice if you know a way around the above. However, if you don't, and still want to contribute, developing a way to seamlessly call the DecompilerInterface class remotely (started in DecompilerSever implementation), that could solve this entire issue. I got stalled on it because of issues getting the client to behave that same as using a normal local DecompilerInterface.

If you are interested in helping (like @xeniarose :D), feel free to ping me for more info. Any form of help is appreciated <3.

[mahaloz]

TODO

• Fix enums syncing (breaks on mahaloz_main)
• Fix auto-connect to the right socket
• Fix client not shutting down after disconnecting from server

[mahaloz]

WE BALL ITS DONEEEEEEEEEEEEEEEE

[k4lizen]

LETS GOOOO GOOD JOB!!