[DAEMON-320] appcd-plugin: Invoke external plugins using workers
DescriptionAppc Daemon plugins support internal and external plugins. External plugins are run in a freshly spawned Node.js process. The external plugin communicates with the appcd core using an IPC tunnel. One problem is serializing and deserializing data is bottlenecking throughput. By switching to Node.js 12, we can use the new worker thread library (https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html). Since worker threads run in the same process, we can take advantage of shared memory and that should be much, much faster. Another advantage is the worker API allows us to terminate the worker and free up the memory, which is one of the biggest benefits of running a plugin in a subprocess. We need to investigate to make sure workers cannot accidentally call
process.exit()or something and take down the core. Furthermore, we need to make sure we can gracefully handle a plugin with bad syntax. While workers solve many problems, we still need to keep the current design just in case a plugin needs to lock down the Node.js version.