Storing Data in Node-Red Variables

Node-red nodes pass the msg object between nodes.

However this object is replaced by the next msg object. So how do you store data between node calls?

Node-Red provides three mechanisms:

  • The context object -stores data for a node
  • The Flow object – stores data for a flow
  • The global object -stores data for the canvas





I will illustrate these methods using a simple flow like the one below.simple-node-flow

The inject node is used to start the flow , then function node implements the counter and the debug node displays the result.

The idea is to count the number of times the message was injected.

The actual message that gets injected isn’t important.

Note: There is a video to accompany this tutorial as it is easy to demonstrate the interaction between variables using video. You may want to read quickly through the text and then go to the video. Here is the video

Using the Context Object

This is used for storing function variables.

The process for retrieving and storing is to use the get method of the object for retrieving value and the set method to store values.:

name =context.get("name"); //to retrieve a variable  
context.set("name",name); // to store a variable

A variable stored for function 1 in the context object is not available to function 2 and vice versa.

context-variables-node-red-flow

Initialising the Variable

The standard way is to include this code at the top of the script:

var count=context.get('count') || 0;

Which means- If count doesn’t exist in the context object then make our local variable count zero; otherwise assign the stored value to our local variable count.

You can use multiple variables e.g.

var count=context.get('count') || 0;
var count2=context.get('count2') || 0;

You can also use an array or an object e.g

var local=context.get('data') || {};
if (local.count===undefined) //test exists
{
  local.count=0;
}

In the code above data is an object stored in the context object and local is our local object.

Here is an example script that uses a single variable as a counter in function 1:

var count=context.get('count') || 0;
count +=1;
msg.payload="F1 "+msg.payload+" "+count;
context.set('count',count);
return msg;

Note the above syntax is commonly seen on the Internet but it does cause strange problems due to the way JavaScript identifies True and False.

There instead of

var count=context.get('count') || 0;

I prefer to use this format:

var count=context.get('count') ;
if (typeof count=="undefined")
count=0;

Here is an example script that uses a object variable as a counter in function 2::

var local=context.get('data') || {};
if (local.count===undefined)//test exists
{
  local.count=0;
}
local.count +=1;
msg.payload="F2 "+msg.payload+" "+local.count;
context.set('data',local);
return msg;

If you look at the code for function 1 and function 2 you will see that they use the same counter variable..

However when you click the inject node for function 1 you see the counter value is 1 and then the inject node for function 2 then counter is also 1.

This shows that the counters are local to each function and not shared between functions.

Note: Currently if you restart the flow the variables are reset but this is likely to change.

Using the Flow Object

You use the flow object in the same way as the context object.

To retrieve values stored in the flow object use:

var count=flow.get('count') || 0;

and to store values use:

flow.set('count',count);

This time you should notice that the functions can share variables stored in flow objects. See video

Using the Global Object

You use the flow object in the same way as the context object.

To retrieve values stored in the flow object use:

var count=global.get('count') || 0;

and to store values use:

global.set('count',count);

This time you should notice that the functions can share variables stored in the global object even across flows.

 Video -How to Store Data in Node-Red Variables

Flows Used in the video

Storing and Retrieving Multiple Variables

In node-red version 0.19 it became possible to store and retrieve several variables at once. So instead of using:

var v1=flow.get("v1");
var v2=flow.get("v2");
//You can use
var values=flow.get(["v1","v2"]);
var v1=values[0];
var v2=values[1];
//and
flow.set("v1",v1);
flow.set("v2",v2);
//You can use
flow.set(["v1","v2"],[1,2]);

If you look at the variables in the store you can see that they are stored as individual variables as  shown in the screen shot below. The screen shot also shows how an variable array (data) is stored:

multiple-variables

Storing Context Data in The File System

Data stored in the context,flow and glbal variables in known as context data and it is normally stored in memory.

This means that if you restart the flow then the data is lost.

However from version 0.19 it is possible to store the context data in the file system.

In order to do this you will need to modify the settings file and add the following entry:

contextStorage: {
   default: "memoryOnly",
   memoryOnly: { module: 'memory' },
   file: { module: 'localfilesystem' }
},

It doesn’t matter where in the settings file you place it and depending on the version of node-red you started with you may already have an entry that is commented out.

The settings above configure node-red to use the default store in memory and also the filesystem for the filestore.

We therefore have two stores.

When saving data to the context variables or retrieving data from them you will need to specify the store they are in. The default is in memory.

So if you use:

var count=context.get("count");

You will retrieve the count variable from memory and to get the count variable from the file store use:

context.get("count", "file");

To store data use:

context.set("count", count,"file");

The system stores the variables in a JSON file in a folder called context under the .node-red folder.

Even though you are storing data in the file system it is still possible to loose data as the data is only flushed to the file system every 30 seconds.

You can change this (no real need) and other configuration options see the docs here.

You should also note that you can have two variables with he same name but stored in memory and filesytem as shown in the screen shot below:

store-context-data-filesystem

Exchange Data Between Nodes Without Wires

With node-red messages are usually passed between nodes using wires.

However it is possible to pass data between nodes with them being wired together using flow and global objects.

Video- Moving Messages between Flows Without Wires

Related tutorials

Please rate? And use Comments to let me know more
[Total: 19   Average: 4.5/5]

27 comments

  1. At the beginning you wrote: context.set(“name”)=x; // to store a variable

    All other examples say: context.set(“count”, count);

    I think the first one is not correct.

  2. This Tutorials are by far the best. THX alot for your work.

    Short Question. Exist a smart way to store payloads into arrays?
    I wann save the current state of my lights so Im able to reset them after using the “clean the room”-modus
    or sync the lights. When i am going to change the Mater I want the others to follow.
    The topics contain allready the array ID within their Light name.

    1. Hi
      Can you give me a few more details. What data structure are you using at the moment to store the light status. Is it simple like on/off or do you have other settings? etc

      1. wow, this is a fast reply.

        I would like to store more or less the hole payload of a light.
        on/off colour status colourtemp.

        So i actually plan to just have an arry of these objects. Couse I’m relativly new to this I just own two yeelight lights and two double switches and waiting for some more. I will try to programm a smal example. Maybe some pictures will help. I would love to get some feedback afterwards.

        Thx alot steve your examples helped me very very mutch to get an understanding of the stucture flow and syntax.

      2. // After some try and error I go for this result
        var stringID = msg.topic.substring( msg.topic.length-3) + “1”; //last 3 digits + “1” ;
        var ID = parseInt(stringID);

        const storagelength = 2;
        var Lock = global.get(‘Lock’) || false; // maybe not Global

        if(Lock) // Storage is Locked
        {
        return null;
        }
        else if( (ID === 0) || (ID >= storagelength) )
        {
        msg.payload.Error = “Unexpected ID while storing”;
        return msg;
        }
        else
        {
        ID = ID/10;
        var Storage = [];
        Storage = global.get(‘Storage’) || [storagelength]; // load
        msg.payload.ID = ID;

        Storage[ID].state = msg.payload.state;
        Storage[ID].brightness = msg.payload.brightness;
        Storage[ID].color_temp = msg.payload.color_temp;

        global.set(‘Storage’,Storage); // safe
        //return msg;
        }
        return null;
        //
        With this function i can store my lights status maybe I will switch to storing complete msg so I know more about the topic and can reset the lights after Locking it with a foreach loop. As i said at the moment I just have 2 lights but Im locking for some more and even some smart sockets or rgb lights will be able to add very easy // hopefully.
        If you have some critc or adjustments plz let me know.
        thx and have a nice weekend

  3. Hi Steve, great content.
    What I am struggling with (and I can imagine others do too) is implementing a flow-wide variable that only applies to a specific source device (sensor etc.).
    So that if you have multiple devices, each would have its own counter, for instance.
    The Node-RED implementation I am using does provide msg.messageOriginDevice.networkId, which could be used as a unique identifier, but any ideas on how that could be worked into a flow (or global) variable?
    Thanks, and keep up the good work, Max

    1. Max
      I would use an object like
      {sensor1:variable1,sensor2:variable2,}
      if you have several variables you can use a nested array or object:
      {sensor1:[variable1,variable2]}
      {sensor1:{variable1:value1,variable2:value2},sensor2:{variable1:value1,variable2:value2}}
      Does that help
      Rgds
      steve

  4. Steve, mind forwarding that sample flow you sent to Riaan to me as well? I am trying to do the EXACT same thing and pulling my heir out. Would love to see your solution. I want to receive an alert if the rate of change is X to detect when people are taking a shower, based on a humidity sensor.

  5. Once again: Steve to the rescue.
    I have a NR flow that gets local weather info from the National Weather Service, packages it into a web page and FTPs it to a page that is served on my domain (why? learning new stuff). I installed a Sonoff TH16 to turn my porch fairy lights on/off and also to sense very local temp/humidity re-flashed with Tasmota and MQTT’d to my RPi broker. How to get the local temps into the web page when the weather NWS data pull and temp sampling are asynchronous? Use a flow variable! And you showed me how they worked.
    Every minute I get temp/humidity and stick it in a flow variable. Every minute I get data from NWS and incorporate the flow variable into the web page. It’s like magic.
    I have your page bookmarked because there is so much stuff here, all explained well. Thanks.
    –Jeff

  6. Hi Steve,
    thank you for your detailed explanations.
    To initialize an undefined flow variable, i have been using the syntax “var count=flow.get(‘count’) || 0;” without a problem for some time.
    But just now i have a flow where my flow-variable is always set to the initialized value, rather than the saved value. Leaving off the “|| 0” will return the saved value. Testing with “undefined” is also working correctly.
    So i am wondering if this syntax with the “|| 0” is really reliable? It is just an OR – how can i be sure that node-red takes the left side first?
    Regards,
    Peter

    1. Peter
      I’ve experienced similar problems and hope to put together a short tutorial on it as it can be very confusing.
      It is related to how Javascript evaluates true and false.
      What I prefer to do is use an init function that runs at the start and creates the variables with known initial values.
      Also using
      var var1=flow.get(‘var1’);
      if (var1===undefined){
      set initial values
      }
      I think works better

      rgds
      steve

  7. In the case of variables in flow scope, what happens if a second message arrives and updates the same flow scope variable, before the first message has been fully processed? Can arrival of second message impact flow of first message because of a variable value change?

    1. I haven’t tested it but I would think not. When the first message arrives, unless it does some asynchronously, then it is processed to the end before node can pick up the other message.
      Node might be receiving messages from MQTT in the background but they are processed by the flow in order.
      Do you a particular example in mind?

  8. Hi could you cc me that flow ? i’m working on a battery temp sensor arrangement to warn if i get overheating . using 4 sensors trying to figure out how to use a global array variable to store the values in.
    thx

  9. Hi Steve,

    I’ve read through your tutorial on how to store data, and I think part of this will apply to my project. What I need to do is store and compare temperature data. I am already receiving temperature data from a sensor (about every 10 seconds) which is then fed into Node-Red (via MQTT connection), changed, and then inserted into a database. I want to receive an alert if the rate of change is X (say if temps rise by 1 degree per minute). I’m new to programming and Node-Red specifically and would appreciate guidance/help on this matter. Have done the alert/notification part via email, but have no idea how to go about setting up a flow that will store and compare temp changes.

    Regards
    Riaan

Leave a Reply to Riaan Cancel reply

Your email address will not be published. Required fields are marked *