Debug Adapter Protocol (DAP)

Request For Comments
#1

Hey all,
Taking inspiration from some of the discussions in this Discourse group about the built-in IDE of SWI-Prolog in comparison to GNU Emacs and other editors, I’ve decided to take a shot at implementing a Debug Adapter Protocol server for SWI-Prolog with the hopes of bringing the built-in graphical tracer’s strengths into my usual Emacs based workflow.

A very partial implementation can be found here, it’s far from ready but I feel the structure of the server is pretty much in place, so I figured I’d better share it already to get some pointers from the gurus.

One thing in particular that I was wondering is how I can have the server notified whenever a debugged thread spawns a new thread, this could be useful for implementing DAP’s Thread event.
Is there a way to hook thread_create/2 perhaps?

Anyhow here’s a quick demo session using GNU Emacs with dap-mode stepping through a simple program:
ezgif.com-gif-maker
Check out dap-swi-prolog.el for the dap-mode extension.

Cheers :slight_smile:

4 Likes
#2

Seems the system only allows you to be notified if a thread completes. Will have a look.

edit: Added channel thread_start to prolog_listen/2.

2 Likes
#3

One potential problem I ran into when using both hooks and prolog_trace_interception/4 is that the messages can be out of sync. Since there is no time stamp or such associated with each message I found no way to put them in order. Also, even if they have timestamps, there is no way to know how long to wait for an event from one before considering that the event will not happen.

This idea from Jan W. still gets my vote.

If we want programmatic access I guess we should provide a proper API for that.

#4

Great! that’s exactly the solution I was hoping for, I’ll build the new revision of SWI-Prolog later today and try it out

#5

For those wanting to check out the code changes, see: ADDED: prolog_listen/2,3: listen to thread create events. · SWI-Prolog/swipl-devel@fd9980b · GitHub

#6

I’ve finally managed to get percise in-clause source positions, like you’d get with the built-in graphical debugger.
It was actually quite straightforward after reading the source code of pce_prolog_tracer a few… hundred times :slight_smile:

I still haven’t implemented support for showing decompiled source code for dynamic predicates, which is up next on my TDL, but other than that it should be usable for visually stepping though the execution of goals.

Demo of stepping through a goal execution with in-clause source positions

ezgif.com-gif-maker(1)

I’ve also implemented a simple DAP client library, which consists of a predicate dap_request_response that sends a request to a DAP server, and awaits a response for a given timeout while aggregating and optionally handling DAP events via a given callback.
The client library is utilized in the server’s test-suite to emulate user interaction with a DAP enabled IDE talking to the server.
The test-suite itself is also a work in progress, notably I couldn’t find any standard comprehensive test suite for DAP servers yet.

PS If anyone feels like trying it out, I’d really appreciate letting me know of any issues/requests :slight_smile:

3 Likes
#7

Btw, if you want some other code to reference for finding source positions and such, my LanguageServer might be of use.

#8

I’m trying to use this, but I get "Wrong type argument: stringp, nil" when I try to dap-debug a goal. It’s entirely possible I’m doing something wrong on my end though, I’ve never used dap-mode before.

#9

Actually, the initial stdio handling in DAP server was a copy-paste of the stdio_server from lsp_server :slight_smile:

Alright!
I’m not a dap-mode expert either, but let’s see… does the minibuffer suggest SWI-Prolog Run Configuration after you hit M-x dap-debug RET?
Screen Shot 2021-09-25 at 18.58.31

Also, can you check that Emacs is able to find the debug_adapter built file?

(locate-file "debug_adapter" exec-path exec-suffixes 1)

If the above snippet returns nil, you should try to set the value of dap-swi-prolog-debug-program to point to the location of debug_adapter.

1 Like
#10

:smiley: Excellent!

I do indeed get the SWI-Prolog Run Configuration option, but subsequently entering the goal fails with the error.

That was returning nil, but after setting dap-swi-prolog-debug-program, I still get the same error.

#11

OK, their are a couple more places I’d look for answers:
First, running M-x toggle-debug-on-error should cause Emacs to print more information so we can see what variable is expected to be a stringp but ends up nil.

Also, each time you run dap-debug, dap-mode runs the external server and logs its stdout and stderr to an Emacs buffer called *SWI-Prolog::Run std{err,out}* or *SWI-Prolog::Run<N> std{err,out}* (where N is a number) if you’ve ran multiple sessions. By default debug_adapter enables the debug topic dap(tracer) which should minimally emit a line like:

% [Thread 3] tracer setup

So if you see this line in *SWI-Prolog::Run stderr* that’ll ensure that the tracer successfully loaded the buffer you were visiting in Emacs and started a thread to run the given goal.

dap-mode also has a useful transaction log, setting the variable dap-print-io to t will make dap-mode log all of the DAP messages sent and received as plain JSONs to the *Messages* buffer.

#12

I actually tried doing that, but it didn’t actually get a backtrace…which I suppose means that the error is being caught elsewhere and it’s just printing out the message after the fact? Very puzzling.

Additionally, I don’t see either of the run buffers…puzzling.

I am running Emacs 28, built off master a few months ago, so entirely possible something is just weird in my setup.

I’m also about to try updating my emacs & packages; I’ll see how it works after that & try to debug further & report back/on the github; I would very much like to get this working!

#13

Ah, figured it out! It seems that :dap-server-path needs to be a list; the error was coming from make-process, when the command was a string, instead of a list

1 Like
#14

Nice!
Thank you for debugging and for the PR :slight_smile:

In the mid time I’ve also added support in for showing decompiled dynamic clauses in the tracer, as well as for the continue DAP request which is similar to f(inish) in the built-in graphical debugger. continue can be sent by dap-mode with M-x dap-continue or by typing c with dap-hydra active.

2 Likes
#15

Can you also do Prolog’s secret weapon, retry?

#16

Certainly! added in Added support for the `restartFrame` DAP request by eshelyaron · Pull Request #6 · eshelyaron/debug_adapter · GitHub.

Demonstration of retrying a frame

ezgif.com-gif-maker

3 Likes
#17

Looked at tracer.pl and felt like I was looking at some old and familiar code.

I know there is a lot of specialized knowledge of SWI-Prolog internals captured as code in there but many will not understand it. Perhaps if you made a separate version with copious comments it would help others. I know there are a few things in there that I will have to review more to understand. :slightly_smiling_face:

#18

Hi again,
I’ve published debug_adapter as a pack, so now running pack_install(debug_adapter) will install the debug adapter executable in a sensible location with the name swipl_debug_adapter (Note the name of the executable changed from simply debug_adapter to avoid future collisions) .

One thing I wanted to point out as I think it may be useful for others, is that after some struggling I’ve got the packaging of the pack to use autotools to only specify the version of the package in a single location, and have it populate the pack.pl file, the name of the generated .zip, and be available at runtime through swipl_debug_adapter -V.
The “trick” is to build pack.pl from a pack.pl.in template, like you’d do with a config.h in a C project, and also to add the produced pack.pl to the list of source files when makeing the saved state for swipl_debug_adapter so the definitions in pack.pl like versioning information are available at runtime.
Probably there’s some cleaner way to get this done that I couldn’t find, but this works pretty well and now I only need to change one line to bump versions.

EDIT: A note about shadowing built-in version/1

Note that adding pack.pl to the saved state shadows the built-in version/1 predicate which may lead to surprising results if we’d actually want to use the built-in version/1 to alter the behavior of version/0

?- version(foo).
true.

?- version.
Welcome to SWI-Prolog (threaded, 64 bits, version 8.3.29)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

foo

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

true.

?- consult('pack.pl').
true.

?- version(N).
N = '0.1.8'.

?- version(foo).
false.
#19

I only specify the version in the pack.pl file and use Github archive production to create the .zip file. Here is the script:

#!/bin/bash

version=`grep ^version pack.pl | sed "s/version('\(.*\)')./\1/"`
download=`grep ^download pack.pl | sed "s/download('\(.*\)')./\1/"`
pack=$(basename $(pwd))

echo $download

confirm ()
{ if [ "$yes" = yes ]; then
    return 0
  fi

  while true; do
    printf "$1"
    read answer
    case "$answer" in
          y*)   return 0
                ;;
          n*)   return 1
                ;;
          *)
                echo "Please answer yes or no"
                ;;
    esac
  done
}

if ! confirm "Package and upload $pack, version $version? (y/n) "; then
  exit 1
fi

if confirm "Tag repository with V$version? (y/n) "; then
  git tag -s V$version -m "Pack release tag for $version"
  git push --tags
fi

case $download in
   *.git) githuburl=$download
          ;;
   *)     githuburl="https://github.com/JanWielemaker/$pack/archive/V$version.zip"
          ;;
esac

if confirm "Install from $githuburl? "; then
swipl << _EOF_
catch(pack_remove($pack), _, true).
pack_install('$githuburl', [interactive(false)]).
_EOF_
fi
2 Likes
#20

A couple more updates:
First, the ELisp code that introduces swipl_debug_adpater to dap-mode is now merged into dap-mode's master, so debugging SWI-Prolog goals with GNU Emacs and dap-mode will be available (almost) out of the box :slight_smile:

Also, I’ve added initial support for providing runtime variable bindings information via the scopes and variables DAP requests. DAP has a nice approach IMO to hierarchical data: the protocol allows the debug adapter server to specify that a value is structured (that’s the terminology used in the specs) and return the “parent” value alone along with a reference for the client to later fetch further information about the value of the “children”.
I thought this design can works nicely when applied to Prolog compound terms, so when providing the value of a compound term swipl_debug_adapter returns only the functor/arity of the term and hints the IDE to fetch the compound’s arguments if and when the user chooses to “expand” the value via some GUI action.
Here’s a screenshot for demonstration, debugging the goal foo(baz,bar(baz(first,second,third,fourth))):

swi-prolog
swi-prolog1492×1186 122 KB

2 Likes