Internal and External modules

A modules is a standalone piece of code composed by a backend part and/or frontend part.

The frontend part of a module is executed on a users browser, but the backend part requires a server to run on.

STLSe gives you the option to run your backend module either on the STLSe infrastructure as a "internal" module, or you can connect your own server through GRPC and have an "exernal" module.

To summarize:
Internal Module is a module that is running on the STLSe infrastructure
External Module is a module that is running on your own server and connects to STLSe through GRPC calls

Internal Module consideration

When a module is uploaded and handled by the STLSe infucrastrture it is executed in a sandbox environment that does not have many of the NodeJS default functions and packages. For example: "util", "fs", "http", ..etc.

For this reason when we create a module that we intend to upload and use as internal module we should add polyfills to the default functions of NodeJS that we intend to use and are not already present.

Out of the box, the following are already present:

• atob/btoa
• Buffer
• BSON
• express

Other Restrictions:

• setInterval is not (yet) available
• setTimeout has a max limit (currently 1 min)
• fetch() and other network capabilities are not available but you can use the use_backend_action.stlse_request hook instead
• Local file system is not available
• There is a limit of total memory allocated

These restriction are in place to allow everyone to deploy the module and run it completely for free.
There is the option to upgrade the account an remove/increase some of the limits.

Benefits:

• Modules are automatically scaled based on demand
• Lower latencies when executing hooks due to all running within the same infrastructure
• An included versioning system that allows you to use simultaneously modules of different versions

When to use:
Internal module are a viable solution for 80%-90% of tasks that a webapp/website may do, it should be the default option and if you need specific requirements from your module that are not possible due to the restriction, then you can switch to the external module deployment

How to deploy:
You need to create a zip file with the following structure:

frontend/
-- module-entry.js
backend/
-- module-entry.js

It requires at least the frontend or backend folder, and it should contain a file named module-entry.js.

After you've created this zip file, you can head over to the STLSe Admin Panel where you can upload an internal version by selecting the zip file specified.

External Module consideration

An external module is hosted on server outside of STLSe and connected via GRPC, meaning that you are free to develop in whatever language/framework you prefer and use any api that are available. There is no difference to the user in how an internal module behaves compared to the external one.

Drawbacks of using external module:

• Requires a server and domain connected to the server in order to host an external module
• Need to manually handle server certificates and version updates
• Cannot deploy different module version in parallel without having a secondary server with the older desired version
• Need to manually implement scalability for high traffic
• If your module is connected to crucial hooks and is not reachable it could block the whole application from being used due to not being able to fully execute the crucial hook

Benefits:

• No restrictions/limits for api usage from your machine
• Is not counted towards the module processing time limits of your account
• Gives you 100% of customizability for narrow use cases that are not possible with the internal module

When to use:
External modules have no restrictions of usage so when the Internal Modules restriction are limiting your use case, you should switch to the external module.
Some examples:

• Make api calls from the frontend directly to your external modules for use cases where you have to upload/download raw files.
• Redirect all database queries to your own desired custom database
• Process intense/heavy load application which may include video processing, file generations, or other computation
• Instead of javascript you can use your own custom language/frameworks as the GRPC connection is language agnostic

How to deploy:

1. You need to have:

   • a server that you can open ports and is accessible from the internet
   • a domain connected to that server

2. Create the external deployment of the module with the following information:

   • Backend host: should be the domain that points to your server, e.g. my_module.stlse.com
   • Backend port: it will be the port that will be used to listen for GRPC request from the server, e.g. 39102
   • Frontend path: it is the path where you can find the module-entry.js file of the frontend part of the module, e.g. https://my_module.stlse.com/frontend/module-entry.js

3. Before building your module make sure that you have genearted a secret from the deployment that you've just created and you place this secret in the remote field in your environment.prod.ts file.

4. After you build your module you will have:

   • the frontend part which should be exposed through nginx/apache, e.g. https://my_module.stlse.com/frontend/module-entry.js
   • the backend part which you should execute with node, e.g. node module-entry.js

5. If all is configured properly when you launch your backend part you should see a successfull GRPC connection, and if you open the frontend path you should see the module-entry.js file of your module. If that is the case then you can add your module to an application and start using it!

Note: It is possible to host the backend module by using the free tunnel subdomain .tunnel.local, but this approach is heavily discourage due to potential network issues and other factors that may make the tunnel unreliable, for this reason we recommend to use tunnelling only for development and testing purposes!