Storing Data in Node-Red Variables

Node-red nodes pass the msg object between nodes.

However this object is replaced by the next msg object. So how do you store data between node calls?

Node-Red provides three mechanisms:

  • The context object -stores data for a node
  • The Flow object – stores data for a flow
  • The global object -stores data for the canvas





I will illustrate these methods using a simple flow like the one below.simple-node-flow

The inject node is used to start the flow , then function node implements the counter and the debug node displays the result.

The idea is to count the number of times the message was injected.

The actual message that gets injected isn’t important.

Note: There is a video to accompany this tutorial as it is easy to demonstrate the interaction between variables using video. You may want to read quickly through the text and then go to the video. Here is the video

Using the Context Object

This is used for storing function variables.

The process for retrieving and storing is to use the get method of the object for retrieving value and the set method to store values.:

name =context.get("name"); //to retrieve a variable  
context.set("name",name); // to store a variable

A variable stored for function 1 in the context object is not available to function 2 and vice versa.

context-variables-node-red-flow

Initialising the Variable

The standard way is to include this code at the top of the script:

var count=context.get('count') || 0;

Which means- If count doesn’t exist in the context object then make our local variable count zero; otherwise assign the stored value to our local variable count.

You can use multiple variables e.g.

var count=context.get('count') || 0;
var count2=context.get('count2') || 0;

You can also use an array or an object e.g

var local=context.get('data') || {};
if (local.count===undefined) //test exists
{
  local.count=0;
}

In the code above data is an object stored in the context object and local is our local object.

Here is an example script that uses a single variable as a counter in function 1:

var count=context.get('count') || 0;
count +=1;
msg.payload="F1 "+msg.payload+" "+count;
context.set('count',count);
return msg;

Note the above syntax is commonly seen on the Internet but it does cause strange problems due to the way JavaScript identifies True and False.

There instead of

var count=context.get('count') || 0;

I prefer to use this format:

var count=context.get('count') ;
if (typeof count=="undefined")
count=0;

Here is an example script that uses a object variable as a counter in function 2::

var local=context.get('data') || {};
if (local.count===undefined)//test exists
{
  local.count=0;
}
local.count +=1;
msg.payload="F2 "+msg.payload+" "+local.count;
context.set('data',local);
return msg;

If you look at the code for function 1 and function 2 you will see that they use the same counter variable..

However when you click the inject node for function 1 you see the counter value is 1 and then the inject node for function 2 then counter is also 1.

This shows that the counters are local to each function and not shared between functions.

Note: Currently if you restart the flow the variables are reset but this is likely to change.

Using the Flow Object

You use the flow object in the same way as the context object.

To retrieve values stored in the flow object use:

var count=flow.get('count') || 0;

and to store values use:

flow.set('count',count);

This time you should notice that the functions can share variables stored in flow objects. See video

Using the Global Object

You use the flow object in the same way as the context object.

To retrieve values stored in the flow object use:

var count=global.get('count') || 0;

and to store values use:

global.set('count',count);

This time you should notice that the functions can share variables stored in the global object even across flows.

Video -How to Store Data in Node-Red Variables

Flows Used in the video

Storing and Retrieving Multiple Variables

In node-red version 0.19 it became possible to store and retrieve several variables at once. So instead of using:

var v1=flow.get("v1");
var v2=flow.get("v2");
//You can use
var values=flow.get(["v1","v2"]);
var v1=values[0];
var v2=values[1];
//and
flow.set("v1",v1);
flow.set("v2",v2);
//You can use
flow.set(["v1","v2"],[1,2]);

If you look at the variables in the store you can see that they are stored as individual variables as shown in the screen shot below. The screen shot also shows how an variable array (data) is stored:

multiple-variables

Storing Context Data in The File System

Data stored in the context,flow and global variables is known as context data and it is normally stored in memory.

This means that if you restart the flow then the data is lost.

However from version 0.19 it is possible to store the context data in the file system.

In order to do this you will need to modify the settings file and add the following entry:

contextStorage: {
   default: "memoryOnly",
   memoryOnly: { module: 'memory' },
   file: { module: 'localfilesystem' }
},

It doesn’t matter where in the settings file you place it and depending on the version of node-red you started with you may already have an entry that is commented out.

The settings above configure node-red to use the default store in memory and also the file system for the file store.

We therefore have two stores.

When saving data to the context variables or retrieving data from them you will need to specify the store they are in. The default is in memory.

So if you use:

var count=context.get("count");

You will retrieve the count variable from memory and to get the count variable from the file store use:

context.get("count", "file");

To store data use:

context.set("count", count,"file");

The system stores the variables in a JSON file in a folder called context under the .node-red folder.

Even though you are storing data in the file system it is still possible to loose data as the data is only flushed to the file system every 30 seconds.

You can change this (no real need) and other configuration options see the docs here.

You should also note that you can have two variables with the same name but stored in memory and file system as shown in the screen shot below:

store-context-data-filesystem

If you have multiple context stores then when using nodes like the change node you will be presented with a choice as shown below:

context-store

However you may want to only use the file store and to avoid confusion it is then easier to only enable that option by removing then memory option in the settings.js file as shown below:

contextStorage: { default: "file",
file: { module: 'localfilesystem' } },

You should be very careful when using multiple stores as it makes flows difficult to debug.

Exchange Data Between Nodes Without Wires

With node-red messages are usually passed between nodes using wires.

However it is possible to pass data between nodes with them being wired together using flow and global objects.

Video- Moving Messages between Flows Without Wires



Related tutorials


Click to rate this post!
[Total: 8 Average: 5]

50 comments

  1. Hi Steve, sorry for my English, I only speak Spanish and I am using a translator.
    Basically I want to use a context variable to store an object (eg: variable.data) and then retrieve a value to use it in another process without altering the original saved.
    If I do it with the change block it works, but if I do it with a script it doesn’t: (initial value of flow.variable.data is 100)

    msg.payload = flow.get(“variable”);
    msg.payload.data = 200;
    return msg;

    then the value of flow.variable.data becomes 200.
    I attach an example:
    Saludos
    Ricardo

    [{“id”:”8dcc4bdaf2ea176d”,”type”:”inject”,”z”:”aedc53fa20ccc3cb”,”name”:””,”props”:[{“p”:”payload”}],”repeat”:””,”crontab”:””,”once”:false,”onceDelay”:0.1,”topic”:””,”payload”:”100″,”payloadType”:”num”,”x”:250,”y”:860,”wires”:[[“fea0bb4da08b33a6”]]},{“id”:”fea0bb4da08b33a6″,”type”:”change”,”z”:”aedc53fa20ccc3cb”,”name”:””,”rules”:[{“t”:”set”,”p”:”variable.dato”,”pt”:”flow”,”to”:”payload”,”tot”:”msg”}],”action”:””,”property”:””,”from”:””,”to”:””,”reg”:false,”x”:540,”y”:860,”wires”:[[“506cf08d111c69b2”]]},{“id”:”506cf08d111c69b2″,”type”:”debug”,”z”:”aedc53fa20ccc3cb”,”name”:”debug 31″,”active”:true,”tosidebar”:true,”console”:false,”tostatus”:false,”complete”:”false”,”statusVal”:””,”statusType”:”auto”,”x”:780,”y”:860,”wires”:[]},{“id”:”d1d340852746341f”,”type”:”function”,”z”:”aedc53fa20ccc3cb”,”name”:”funcion”,”func”:”msg.payload = flow.get(\”variable\”);\nmsg.payload.dato = 200;\nreturn msg;”,”outputs”:1,”noerr”:0,”initialize”:””,”finalize”:””,”libs”:[],”x”:500,”y”:900,”wires”:[[“3765fbbb0d6c4b43”]]},{“id”:”3765fbbb0d6c4b43″,”type”:”debug”,”z”:”aedc53fa20ccc3cb”,”name”:”debug 32″,”active”:true,”tosidebar”:true,”console”:false,”tostatus”:false,”complete”:”false”,”statusVal”:””,”statusType”:”auto”,”x”:780,”y”:900,”wires”:[]},{“id”:”d7da027042dabd50″,”type”:”inject”,”z”:”aedc53fa20ccc3cb”,”name”:””,”props”:[{“p”:”payload”},{“p”:”topic”,”vt”:”str”}],”repeat”:””,”crontab”:””,”once”:false,”onceDelay”:0.1,”topic”:””,”payload”:””,”payloadType”:”date”,”x”:260,”y”:900,”wires”:[[“d1d340852746341f”]]},{“id”:”84896d58e374d8c1″,”type”:”change”,”z”:”aedc53fa20ccc3cb”,”name”:””,”rules”:[{“t”:”set”,”p”:”variable.dato”,”pt”:”flow”,”to”:”payload”,”tot”:”msg”}],”action”:””,”property”:””,”from”:””,”to”:””,”reg”:false,”x”:540,”y”:700,”wires”:[[“c5bc7d6cbda4cae3”]]},{“id”:”c5bc7d6cbda4cae3″,”type”:”debug”,”z”:”aedc53fa20ccc3cb”,”name”:”debug 35″,”active”:true,”tosidebar”:true,”console”:false,”tostatus”:false,”complete”:”false”,”statusVal”:””,”statusType”:”auto”,”x”:780,”y”:700,”wires”:[]},{“id”:”1940a9af278a496a”,”type”:”debug”,”z”:”aedc53fa20ccc3cb”,”name”:”debug 36″,”active”:true,”tosidebar”:true,”console”:false,”tostatus”:false,”complete”:”false”,”statusVal”:””,”statusType”:”auto”,”x”:780,”y”:740,”wires”:[]},{“id”:”fd54bacb2f17e021″,”type”:”inject”,”z”:”aedc53fa20ccc3cb”,”name”:””,”props”:[{“p”:”payload”},{“p”:”topic”,”vt”:”str”}],”repeat”:””,”crontab”:””,”once”:false,”onceDelay”:0.1,”topic”:””,”payload”:””,”payloadType”:”date”,”x”:260,”y”:740,”wires”:[[“fc7cd3402bd6e118”]]},{“id”:”9ee1c09b11da0bd3″,”type”:”inject”,”z”:”aedc53fa20ccc3cb”,”name”:””,”props”:[{“p”:”payload”},{“p”:”topic”,”vt”:”str”}],”repeat”:””,”crontab”:””,”once”:false,”onceDelay”:0.1,”topic”:””,”payload”:”100″,”payloadType”:”num”,”x”:250,”y”:700,”wires”:[[“84896d58e374d8c1”]]},{“id”:”fc7cd3402bd6e118″,”type”:”change”,”z”:”aedc53fa20ccc3cb”,”name”:””,”rules”:[{“t”:”set”,”p”:”payload”,”pt”:”msg”,”to”:”varible”,”tot”:”flow”},{“t”:”set”,”p”:”payload.dato”,”pt”:”msg”,”to”:”200″,”tot”:”num”}],”action”:””,”property”:””,”from”:””,”to”:””,”reg”:false,”x”:520,”y”:740,”wires”:[[“1940a9af278a496a”]]},{“id”:”b7a9746dce141354″,”type”:”comment”,”z”:”aedc53fa20ccc3cb”,”name”:”the value of flow.variable.data is altered”,”info”:””,”x”:450,”y”:820,”wires”:[]},{“id”:”80114046d6f5b895″,”type”:”comment”,”z”:”aedc53fa20ccc3cb”,”name”:”the value of flow.variable.data is not altered”,”info”:””,”x”:460,”y”:660,”wires”:[]}]

    1. Sorry for the delay but I have been away on holiday. If you still need help let me know and I will take a look.
      Rgds
      Steve

    2. Hi
      I can’t import the flow as it gives an error. Can you use the ask steve page and contact me and then you can email me the flow.
      Rgds
      Steve

      1. Hi Steve ,

        I´ve get the same problem.

        Can it be that objects and araays of objects via context variable are to be considered as call by reference. and is there a workaround here?

        Rgds
        Steve

  2. Can I reference a global variable with a variable?
    I have a global object that need to reference that lists many lights status. I want to reference these lights with a counter.
    var l = global.get(‘status.lights.34.state’)

    I want to replace 34 with a variable.
    Is that possible?

    1. yes
      if your var is called light
      then
      let status=global.get(“status”);
      let state=status.lights[light][“state”];

      should work
      Rgds
      Steve

  3. Hi guys
    I have an issue with an aircon system I check the temp sensors for each room and save them as global variables.
    Then I check if the zones for the system are open I run the unit, and once the temp sensor reaches the setpoint I turn off the unit and put the system in a stand-by loop checking every 15 sec if any zone has dropped
    It works perfectly for a couple of hours but after that, it can read the global variables and gets lost. when I deploy it solves the issue. Is there a limit of ‘saves’ we can store in a global variable?

    please help I have no idea why is doing this thanks

  4. Hi steve,
    i’ve got a question about global variables on node red :
    how can i do to have the list of global variable set?
    In my flow, I use MQTT for take the value of sensor. For exemple I’ve got 2 msg from MQTT.
    {“topic”:”maison/bureau/temperature”,”payload”:”27.90″,”qos”:1,”retain”:false,”_topic”:”maison/bureau/temperature”,”_msgid”:”a1050b42e636b4ae”}
    and
    {“topic”:”maison/bureau/humidity”,”payload”:”41.00″,”qos”:1,”retain”:false,”_topic”:”maison/bureau/humidity”,”_msgid”:”8e12d05712556078″}

    then, I put a node with only this ligne :
    global.set(msg.topic,msg.payload)

    I try global.get(global.keys()) but I havn’t the topic with the value. how can I get the name of the variable and its value?

    1. Hi
      you can’t use the msg object for the global
      you should choose a variable like data and use that
      var data=global.get(“data”) || {};
      data.topic=msg.topic;
      data.message=msg.payload;
      global.set(“data”,data);

      rgds
      steve

  5. Hi Steve,

    I have question.
    If I want to store the nvalue value of this message, which runs through a function node, in a context variable:
    msg.payload = {
    “Battery”: 255,
    “RSSI”: 12,
    “description”: “”,
    “dtype”: “Light/Switch”,
    “hwid”: “8”,
    “id”: “000141AF”,
    “idx”: 351,
    “name”: “TV Power”,
    “nvalue”: 0,
    “stype”: “Switch”,
    “svalue1”: “0”,
    “switchType”: “On/Off”,
    “unit”: 1
    }

    Something like: var dz_nvalue=context.get(‘nvalue’) || 0;
    How do would I do that?

    Greetzzz,

    Gerben

    1. Hi try
      //
      var nvalue=context.get(‘nvalue’);
      If (typeof nvalue==”undefined”) //works better than ||0
      nvalue=0; //set default
      nvalue=msg.payload.nvalue;
      context.set(“nvalue”,nvalue); //save it
      //
      Rgds
      Steve

  6. Hi Steve,

    nice videos and tutorials. I am learning a lot.
    I am quite new to node-RED (and to programming) but is it possible to use this function to retrieve API client_id and client_secret from a file and place it in variables inside the Authorisation node. I would like to make the Authorisation node(s) available for other but ideally without my credentials.
    Thanks
    Klaus

    1. Not quite sure what you mean by authorisation node.I assume you are using http? is that correct. I have seen implementations I think it was a crypto flow that retrieved authorisation codes and used them but the API was designed for that.
      Can you supply more details
      Rgds
      Steve

      1. Thanks, Steve,

        here are the details.

        I am using this to obtain a bearer token for subsequest requests. Ideally I don’t want my credentials hard-coded in the code but preferably access using variables.
        ————————————–

        var loginAPI = “https://api-prd.xxxx.com/oauth/client_credential/accesstoken?grant_type=client_credentials”;

        msg.method = “POST”;
        msg.url = loginAPI;
        msg.headers = {};
        msg.headers[“Content-Type”] = “application/x-www-form-urlencoded”;

        msg.payload = ‘client_id=client_id&client_secret=client_secret’;
        return msg;
        ———————————————-

        It tried to work it out using your variables tutorial but somehow get stuck.
        Thanks
        Klaus

  7. At the beginning you wrote: context.set(“name”)=x; // to store a variable

    All other examples say: context.set(“count”, count);

    I think the first one is not correct.

  8. This Tutorials are by far the best. THX alot for your work.

    Short Question. Exist a smart way to store payloads into arrays?
    I wann save the current state of my lights so Im able to reset them after using the “clean the room”-modus
    or sync the lights. When i am going to change the Mater I want the others to follow.
    The topics contain allready the array ID within their Light name.

    1. Hi
      Can you give me a few more details. What data structure are you using at the moment to store the light status. Is it simple like on/off or do you have other settings? etc

      1. wow, this is a fast reply.

        I would like to store more or less the hole payload of a light.
        on/off colour status colourtemp.

        So i actually plan to just have an arry of these objects. Couse I’m relativly new to this I just own two yeelight lights and two double switches and waiting for some more. I will try to programm a smal example. Maybe some pictures will help. I would love to get some feedback afterwards.

        Thx alot steve your examples helped me very very mutch to get an understanding of the stucture flow and syntax.

      2. // After some try and error I go for this result
        var stringID = msg.topic.substring( msg.topic.length-3) + “1”; //last 3 digits + “1” ;
        var ID = parseInt(stringID);

        const storagelength = 2;
        var Lock = global.get(‘Lock’) || false; // maybe not Global

        if(Lock) // Storage is Locked
        {
        return null;
        }
        else if( (ID === 0) || (ID >= storagelength) )
        {
        msg.payload.Error = “Unexpected ID while storing”;
        return msg;
        }
        else
        {
        ID = ID/10;
        var Storage = [];
        Storage = global.get(‘Storage’) || [storagelength]; // load
        msg.payload.ID = ID;

        Storage[ID].state = msg.payload.state;
        Storage[ID].brightness = msg.payload.brightness;
        Storage[ID].color_temp = msg.payload.color_temp;

        global.set(‘Storage’,Storage); // safe
        //return msg;
        }
        return null;
        //
        With this function i can store my lights status maybe I will switch to storing complete msg so I know more about the topic and can reset the lights after Locking it with a foreach loop. As i said at the moment I just have 2 lights but Im locking for some more and even some smart sockets or rgb lights will be able to add very easy // hopefully.
        If you have some critc or adjustments plz let me know.
        thx and have a nice weekend

  9. Hi Steve, great content.
    What I am struggling with (and I can imagine others do too) is implementing a flow-wide variable that only applies to a specific source device (sensor etc.).
    So that if you have multiple devices, each would have its own counter, for instance.
    The Node-RED implementation I am using does provide msg.messageOriginDevice.networkId, which could be used as a unique identifier, but any ideas on how that could be worked into a flow (or global) variable?
    Thanks, and keep up the good work, Max

    1. Max
      I would use an object like
      {sensor1:variable1,sensor2:variable2,}
      if you have several variables you can use a nested array or object:
      {sensor1:[variable1,variable2]}
      {sensor1:{variable1:value1,variable2:value2},sensor2:{variable1:value1,variable2:value2}}
      Does that help
      Rgds
      steve

  10. Steve, mind forwarding that sample flow you sent to Riaan to me as well? I am trying to do the EXACT same thing and pulling my heir out. Would love to see your solution. I want to receive an alert if the rate of change is X to detect when people are taking a shower, based on a humidity sensor.

  11. Once again: Steve to the rescue.
    I have a NR flow that gets local weather info from the National Weather Service, packages it into a web page and FTPs it to a page that is served on my domain (why? learning new stuff). I installed a Sonoff TH16 to turn my porch fairy lights on/off and also to sense very local temp/humidity re-flashed with Tasmota and MQTT’d to my RPi broker. How to get the local temps into the web page when the weather NWS data pull and temp sampling are asynchronous? Use a flow variable! And you showed me how they worked.
    Every minute I get temp/humidity and stick it in a flow variable. Every minute I get data from NWS and incorporate the flow variable into the web page. It’s like magic.
    I have your page bookmarked because there is so much stuff here, all explained well. Thanks.
    –Jeff

  12. Hi Steve,
    thank you for your detailed explanations.
    To initialize an undefined flow variable, i have been using the syntax “var count=flow.get(‘count’) || 0;” without a problem for some time.
    But just now i have a flow where my flow-variable is always set to the initialized value, rather than the saved value. Leaving off the “|| 0” will return the saved value. Testing with “undefined” is also working correctly.
    So i am wondering if this syntax with the “|| 0” is really reliable? It is just an OR – how can i be sure that node-red takes the left side first?
    Regards,
    Peter

    1. Peter
      I’ve experienced similar problems and hope to put together a short tutorial on it as it can be very confusing.
      It is related to how Javascript evaluates true and false.
      What I prefer to do is use an init function that runs at the start and creates the variables with known initial values.
      Also using
      var var1=flow.get(‘var1’);
      if (var1===undefined){
      set initial values
      }
      I think works better

      rgds
      steve

  13. In the case of variables in flow scope, what happens if a second message arrives and updates the same flow scope variable, before the first message has been fully processed? Can arrival of second message impact flow of first message because of a variable value change?

    1. I haven’t tested it but I would think not. When the first message arrives, unless it does some asynchronously, then it is processed to the end before node can pick up the other message.
      Node might be receiving messages from MQTT in the background but they are processed by the flow in order.
      Do you a particular example in mind?

  14. Hi could you cc me that flow ? i’m working on a battery temp sensor arrangement to warn if i get overheating . using 4 sensors trying to figure out how to use a global array variable to store the values in.
    thx

  15. Hi Steve,

    I’ve read through your tutorial on how to store data, and I think part of this will apply to my project. What I need to do is store and compare temperature data. I am already receiving temperature data from a sensor (about every 10 seconds) which is then fed into Node-Red (via MQTT connection), changed, and then inserted into a database. I want to receive an alert if the rate of change is X (say if temps rise by 1 degree per minute). I’m new to programming and Node-Red specifically and would appreciate guidance/help on this matter. Have done the alert/notification part via email, but have no idea how to go about setting up a flow that will store and compare temp changes.

    Regards
    Riaan

Leave a Reply

Your email address will not be published. Required fields are marked *