Redis Lua benchmark and overhead

I’ve been wanting to make use of Redis Lua scripting for a long time for some projects at work. I’ve read Redis Lua scripting, and found that I can use it to solve a few common issues we’ve been dealing with, one being ensuring redis hashes have an expiry. We have a lot of services at my job which use Redis as a centralized state database, allowing services to share knowledge in a common place, without having to know if or what else may need the data. All information written to this DB requires an expiry value to be assigned. In our old data model, we had done this by using the SETEX [key] [ttl] [value] or SET [key] [value] EX [ttl] commands. This is great, as we can set the key and the expiry at the same time atomically and ensure that every key will expire. In our new data model, we are switching from keys to hashes, but unfortunately, there is not an HSETEX command built in. Instead, we have to rely on pairing HSET and EXPIRE commands manually, which of course doubles the “ops” per item being set, but also ads the possibility of something breaking and a hash lacking expiry.

I found Redis Lua scripting to be a big help here to ensure that every hash gets an expiry set while setting fields/values on the hash. I also added logic to conditionally skip setting EXPIRE. The response of HSET will tell us how many new fields have been created, which would only be 0 if the hash already existed and already had an expiry.

-- Set EXPIRE on hash after HSET if new fields added
-- OPs: usually 2, occasionally 1 (when hash and fields are reused)
local expire = table.remove(ARGV, 1)
local newCount = redis.call("HSET", KEYS[1], unpack(ARGV))
if tonumber(newCount) > 0 then
  redis.call("EXPIRE", KEYS[1], expire)
end
return newCount

So, all of this sounds great, I can just make use of Lua scripts for anything that needs to do multiple actions, saving my code round trips to the DB. Obvious win, right? I wasn’t totally sure. When searching for Redis Lua performance, I kept finding articles where someone was mis-using Redis Lua, calling EVAL every time they want to call Redis Lua, or generating dynamically generated Lua script, both of which are entirely incorrect uses of Redis Lua.

I wanted to know was what is the actual cost was to use a Lua script compared to standard Redis commands? Literally, apples-to-(lua-scripted)-apples.

This script performs a few test redis actions, and then compares it with the equivalent using a Lua script. In order to not have any test affect another, I’m flushing the Redis DB and waiting 5s between each test. Each command is running 25,000 times. I grouped all tests with their Lua equivalents.

Node.js source available at: https://gist.github.com/Brayyy/128394ea44e9748057de310112ef3540

Benchmarking requests per second

--- SET
await SET: 5.393s
await luaSet: 5.440s

--- SETEX
await SETEX: 5.117s
await luaSetEx: 5.606s

--- HSET
await HSET: 5.148s
await luaHSet: 5.299s

--- HSET + EXPIRE
await HSET, blind EXPIRE: 6.052s
await HSET+EXPIRE: 10.147s
await luaHSetEx: 5.353s

--- SETBIT
await SETBIT: 5.641s
await luaSetBit: 5.297s

--- SETBIT + EXPIRE
await SETBIT, blind EXPIRE: 6.019s
await SETBIT+EXPIRE: 10.028s
await luaSetBitEx: 6.709s

Findings

• All O(1) Redis commands vs O(1) Lua scripts performed very similarly. Lua appears to add a small delay, but nothing major. (~4.38% worse)
• All command+expire performed similarly to Lua if we don’t care about waiting for the second command to come back.
• All command+expire performed much worse than Lua if we care about the response of the second command.
• Most of the latency is caused by the Redis Service being on another EC2. This unfairly adds latency to everything, but is more realistic, as running a Redis Service per host is not an option.

When it comes to sheer speed, it seems that my concerns about Redis vs Redis Lua are fairly squashed. In these tests, Lua with a single command performs just as well as a single native command.

System load

It’s not all about req/sec. If the Redis Service is under higher load, the speed gain may not be worth it. I set up a similar script, but this one just runs the HSET with EXPIRE benchmark in a loop until I stop it. I ran it for 30 minutes, waiting for everything to stabilize, and then performed the same test using the luaHSetEx Lua Script. I had already setup Prometheus monitoring with Redis data exporters to make the gathering of stats easy.

Results

SETEX vs luaSetEx

HSETEX vs luaHSetEx

Findings

• Both
  • CPU was largely unaffected comparing native Redis commands with Lua equivalent
  • redis_net_input_bytes_total increased, likely due to the 32-character EVALSHA being longer than the raw commands
• O(1) Redis commands
  • EC2 rx is higher when using Lua, likely due to the 32-character EVALSHA being longer than the raw commands
  • Commands/sec was increased by 100% as EVALSHA counts as an operation/command (EVALSHA+SETEX)
  • redis_commands_duration_seconds_total showed EVALSHA taking up 4x more time than SETEX
• Redis command+expire
  • EC2 rx and tx were both lower when using Lua due to only needing a single round trip
  • Commands/sec was increased by 50% as EVALSHA counts as an operation/command (EVALSHA+HSET+EXPIRE)
  • redis_commands_duration_seconds_total showed EVALSHA taking up 2x more time than the HSET and EXPIRE combined
  • redis_net_output_bytes_total decreased, due to only one round trip

Conclusions

• Lua adds a very minor time delay to the total Redis workload.
• While using Lua as a shim around a single Redis command is pointless, it doesn’t seem to add much latency.
• Lua excels at speeding up multiple round trips, and cutting down on network activity.
• EVALSHA counts toward your total commands/sec. but don’t appear to make much impact on system resources.
• Commands run within the EVALSHA are not masked or hidden away in metrics.
• Don’t be scared to use Lua when you think it makes sense.