Raw to byte or character conversion

#1

Hi,

I just finished my work on a R-client for the BaseX XML-database (see CRAN - Package RBaseX) and am now planning to start working on a similar client in SWI-prolog.
The (all) clients are based on raw non-blocking sockets. All input from the client is converted to a raw stream, the server first responds by returning a zero-terminated raw-stream and then adds either a 0x00 (=succes) or 0x01 (=error). When the input stream is binary, 0x00-bytes and 0xFF- bytes are prepended by an extra 0xFF-byte. They have to be removed from the aoutput stream as returned by the server. When the output-stream represents a character-stream, the raw data has to be converted to character data and care has to be taken of control characters which may be included.

I have two questions:

  • Is there any interest in a Prolog-client?
  • As you can see, transforming and processing the stream is the hardest part of the work. Are there any prolog-modules with handy functions/predicates for transforming raw-streams (add or remove bytes, split stream into other streams etc.)

Ben

#2

You might find these functions useful: contrib-protobufs/protobufs.c at 7fa1c0225e4a58f305f1380aff7c95eef26de7a8 · SWI-Prolog/contrib-protobufs · GitHub

These are low-level conversions between some SWI-Prolog types and bytes, as needed by library(protobufs). They are currently not exported from the protobufs module, but that can be easily changed – you can see how they’re used in protobufs.pl.

If you need these extended a bit, I can probably help you.

#3

Depending on experience, performance impact, etc., I’d consider doing most of the low-level work in C(++). That said, Prolog has all that is needed to read and write bytes. A few things have no portable solution in pure Prolog, such as converting some bytes into a float or converting bytes in unknown native byte ordering into an integer. The stuff in protobufs come handy, either for sharing or getting inspiration.

Otherwise, the more interfaces the better. In the Prolog world SWI-Prolog is doing quite well considering available interfaces. It is still one of the weak points of Prolog though :frowning:

#4

I can read C and C++ but luckily I have always been able to avoid real programmming in one of those languages :wink:. I’ll try but maybe that I will need your help.
Adoption of a new interface is also promoted when a good implementation example is available. Do you have an idea for a good use case?

#5

I am in the same boat.

My advise would be to get the code working correctly with Prolog, then when it works as you seek and if speed or some other checking is needed, then implement portions in C. Also if you are at the C level then consider SIMD. (ref)

#6

Hi Jan,

A C++ interface by a C++ programmer by Volker Wysk post@volker-wysk.de links to a library which dates back to 2004. Is this really the most recent information for interfacing SWI-Prolog and C++?

#7

GCC 2.95 brings back memories … not all of them good. :wink:

This seems to be newer:
https://www.swi-prolog.org/pldoc/doc_for?object=section('packages/pl2cpp.html')

Also, for something as simple as dealing with raw bytes / characters, you might be better off (and slightly more performant) to use the C interface. The protobuf.c file that I mentioned could be a reasonable place to start.

Maybe it would be best if you were to say what “raw byte/character conversion” functionality you want?

#8

Actually, the packaged SWI-cpp.h is the original interface by me. I’m a lousy C++ programmer though. Volker wrote something that is closer to how C++ programmers might want to see Prolog. I have little opinion on that :slight_smile: But yes, the packaged one is maintained and since recently works cleanly using g++-11 on very high warning levels. It doesn’t do much more than a bit of basic type conversion and exception forwarding though. It also doesn’t cover the entire SWI-Prolog C API, but you can freely mix the two in you C++ code.

#9

It’s been 10+ years since I’ve programmed in C++, so I took a quick look at Google C++ Style Guide to remind myself of the many problem areas in that language. There have been a lot of changes in C++ over the years; for example with copyable and movable types, smart pointers, etc. So, it would be good for someone with recent C++ experience to review the C++ interface(s).

One thing I noticed in the “supported” C++ interface - it should probably should be updated to disallow copy constructors and assignment operators unless they make sense (c++ - Why copy constructor and assignment operator are disallowed? - Stack Overflow). There’s more on this here.