REST service and consumer proposal
Open, Needs TriagePublic

Description

Now that I have a better understanding of the system and its limits, I suggest dumping the C++ REST service and its Pistache requirement - those services steal precious resources from a system that should have other priorities. Instead, I would like to propose a VERY slim solution with five main components that are more "secure":

  • cron daemon that calls the bash service at regular intervals
  • bash service that reads / writes to a status cache and can be used by all system files. It should also serve to protect the system from malformed commands, act as a cache-locking mechanism and also enable locking the interface to one user.
  • status cache file that exists on a file-system reserved chunk of ram / or fifo - something like /tmp/status.json
  • netcat GET (:80) / POST (:8080) endpoints (see below for what I mean)
  • offline-first fullscreen web interface (hosted at apertus.org that allows the user to choose a password and local IP address to route its requests to) - can later be recycled for native apps

Here are some starting points for what I mean about using netcat:

# one way to do this
while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat /tmp/status.json; } | nc -l 80; done

# another way
nc -q 1 -w 5 -v -l -p 80 < /tmp/status.json

https://stackoverflow.com/questions/16640054/minimal-web-server-using-netcat

http://www.commandlinefu.com/commands/view/870/sharing-file-through-http-80-port

http://www.commandlinefu.com/commands/view/9164/one-command-line-web-server-on-port-80-using-nc-netcat

nothingismagick updated the task description. (Show Details)
nothingismagick raised the priority of this task from to Needs Triage.
nothingismagick added a subscriber: nothingismagick.

The goal is to avoid to use bash which spawns a lot of threads and to have a central point which should do the configuration and initializations.

Then maybe a single long running bash script with a high priority / nice-value running as a main control interface that listens for changes to the fifo and where necessary modifies the fifo / and non-volatile file-store for use on startup for init and conf...

If you are worried about too many bash scripts, then I am really confused about this folder:

https://github.com/apertus-open-source-cinema/beta-software/tree/master/software/scripts

Daemon was started, amongst other things, to remove the need for that folder at some point in time. Feel free to implement something, which could replace current approach.

Challenge accepted. Can you point me to a resource for a current VM image?

Which VM image?

I thought I read somewhere that there is a virtual machine image that I should use to spin up development... Maybe I imagined that.

It is planned, but wasn't done yet, as we have lack of people, who are involved in practical way.

jatha added a subscriber: jatha.Feb 20 2018, 7:13 PM

There are some things, that are good to consider, when we build something like this:

  • The camera has plenty of resources. When designing something that only works on the parameters, and not on any image data it is almost impossible to design a system that is to imperformant to run on the cam. This means no performance optimisation is needed.
  • The software has to work without requiring internet access at all. (in my opinion) I think it would be handy if everything is served from the camera.
  • The software should manage the state of the hardware (is the fpga configured? what bitstream should be loaded? what bitstream is loaded? what plugin modules are plugged in?
  • New Plugin Modules should be easy to integrate (new bitstrams and new parameter space)
  • An open API should be exposed to allow other interfaces (maybe hardware) to be easily developed

If i remember correctly, then most of this things were discussed with Herbert for the daemon, which was a task for GSoC2017. If you can get it through bash script, fine, but i'm rather sceptical about the performance. Surprise me.

Ok - I've noted what you said @jatha - thanks for the info. Can you tell me where to find the spec for the openAPI and Plugin Modules?

The bash script would just be a convenience wrapper around your other python and shell scripts, adding the opportunity to do low-level writes and reads from the filesystem / ram. If that was something from GSoC2017, is there something that blocked it from integration? GSoC2018 is almost upon us...

It went not well, while GSoC2017, without going too much into details. So i've resumed the work on the daemon, REST server and GUI. The goal is to get rid of scripts to get performance gains. Don't underestimate it, it's an embedded CPU, resources are sparse. There is a reason why C++ was used.

I am seeing two different perspectives here, and just would like to have some clarification - or even better some benchmarking stats. For your information I have done work on arduino, raspberries, Olimex a20-SOM / LIME (including custom flashing of the NAND), Banana-pi and the O-Droid C-1 - so I am VERY aware of paying attention to system resources and the tough design decisions that come with that.


@jatha said: The camera has plenty of resources. When designing something that only works on the parameters, and not on any image data it is almost impossible to design a system that is to imperformant to run on the cam. This means no performance optimisation is needed.

@BAndiT1983 said: Don't underestimate it, it's an embedded CPU, resources are sparse.


Although a bash script is interpreted at runtime and is perhaps a bit slower than python bytecode or c++ - in my opinion using builtins, POSIX language paradigms and well-tested utilities (like e.g. netcat) just seems to me to be better than reinventing the wheel. But I really don't want to get in anyone's way here - just trying to find out how best to contribute.

The ball is on your side. Implement something which fits the requirements and we can test it on the Zynq, to see how it behaves.

So I have "implemented something" - the frontend for the WebRemote and worked with the team today to design a spec. But I would still like @jatha to show some evidence that suggests that "the camera has plenty of resources". I find this hard to believe (its only a relatively low-powered dual arm!!!) and only benchmarks running during a camera state where it is filming and executing scripts could change my opinion. (Or alternatively, a video of htop screen.)

Over three weeks ago I asked for clarification regarding different opinions about "resources", and have yet to receive a reply. Until they are made, I will be siding with @BAndiT1983 on this one - its better to work with little than expect more than you can deliver - because in the former situation you won't ever be disappointed.

perhaps we should change the title of this thread - or move it to "websockets" REST doesn't seem to make sense anymore...