Skip to main content Skip to navigation

Software support forum

Software support forum Debugging tools for mixed Python/C software

You need to be logged in to post in this topic.
  1. Dear Warwick RSE,

    I'm interested in your thoughts on how best to debug software which is written in Python, but on top of compiled code written in C or Fortran. This is something we use/write a lot in my group, but this specific question has arisen from Bug 12753 on Bugzilla.

    Specifically, there is a function in the C part of the software which is called from the Python part of the code. I'd like to put breakpoints into this function, and be able to see a complete backtrace which shows me what route has been taken through the Python part of the software in order to reach that breakpoint. Maybe there's a way to use gdb for this? Alternatively I see interesting things about debugging Python in the cool-and-froody editors that the young people like (Atom, VSCode etc) but can't quite figure out if that functionality stretches to reading symbolic information in compiled codes that Python scripts invoke.

    Any thoughts?

     
  2. In theory, it should be possible to simply invoke the entire python interpreter though gdb, as in

      gdb python

    and then put a breakpoint in the file that you want at the line that you want using

      break filename:linenumber

    before you "run" the python interpreter. So long as you have debugging information compiled into your code it should trigger a stop at the expected point. If you don't have debugging information in the compiled code then it will be a bit trickier because the python interpreter loads the external code using dl_open so the address at which to set the breakpoint is hard to predict.

     
  3. Thanks Chris,

    That kinda-sorta works. I can do

    gdb --args `which python` myscript.py
    (gdb) break mycode.c:3142

    I then confirm that mycode.c relates to a library which hasn't been dynamically loaded yet.

    (gdb) run

    Hits the breakpoint in the C code, and then

    (gdb) backtrace

    Contains a stack trace, but the information that contains is about the function names and source line numbers inside the python interpreter, not inside my Python script. By examining the arguments passed to various functions inside the interpreter I think I can identify which Python file is being executed when my C library function is being called, but not which function or line number within that file called it.

    So useful, but I was hoping for something more Python-aware. Not critically important of course, just wondering if such things exist.

     

     

     

     
  4. Ah, yes. It turns out that there are actually python gdb extensions for dealing with that. I haven't used them myself, but they talk about them a bit at https://wiki.python.org/moin/DebuggingWithGdb

    Sadly that's a system install so you need root on the machine to do it, but it looks like that'l do what you want.

     
  5. Excellent, thanks Chris.

    Now I know that's a thing (I obviously wasn't Googling the right words) I should be able to figure it out from here. I'll look into having the gdb extension activated when loading Python modules on the desktop system.