I’ve been wanting to make use of Redis Lua scripting for a long time for some projects at work. I’ve read Redis Lua scripting, and found that I can use it to solve a few common issues we’ve been dealing with, one being ensuring redis hashes have an expiry. We have a lot of services at my job which use Redis as a centralized state database, allowing services to share knowledge in a common place, without having to know if or what else may need the data. All information written to this DB requires an expiry value to be assigned. In our old data model, we had done this by using the SETEX [key] [ttl] [value] or SET [key] [value] EX [ttl] commands. This is great, as we can set the key and the expiry at the same time atomically and ensure that every key will expire. In our new data model, we are switching from keys to hashes, but unfortunately, there is not an HSETEX command built in. Instead, we have to rely on pairing HSET and EXPIRE commands manually, which of course doubles the “ops” per item being set, but also ads the possibility of something breaking and a hash lacking expiry.

I found Redis Lua scripting to be a big help here to ensure that every hash gets an expiry set while setting fields/values on the hash. I also added logic to conditionally skip setting EXPIRE. The response of HSET will tell us how many new fields have been created, which would only be 0 if the hash already existed and already had an expiry.

1
2
3
4
5
6
7
8

-- Set EXPIRE on hash after HSET if new fields added
-- OPs: usually 2, occasionally 1 (when hash and fields are reused)
local expire = table.remove(ARGV, 1)
local newCount = redis.call("HSET", KEYS[1], unpack(ARGV))
if tonumber(newCount) > 0 then
  redis.call("EXPIRE", KEYS[1], expire)
end
return newCount

So, all of this sounds great, I can just make use of Lua scripts for anything that needs to do multiple actions, saving my code round trips to the DB. Obvious win, right? I wasn’t totally sure. When searching for Redis Lua performance, I kept finding articles where someone was mis-using Redis Lua, calling EVAL every time they want to call Redis Lua, or generating dynamically generated Lua script, both of which are entirely incorrect uses of Redis Lua.

I wanted to know was what is the actual cost was to use a Lua script compared to standard Redis commands? Literally, apples-to-(lua-scripted)-apples.

Read More

I recently purchased some used camera gear, and one of the included items was a “Lens Kit“ by Peak Design, which helps you carry up to two lenses using a strap or on the Peak Design Capture. I always find myself fumbling with a second lens when out, so this seemed like a useful addition to my kit. While I was lucky enough to receive the Canon EF/EF-S version, I have recently switched over to using a Canon EOS R, which uses the new RF mount, which is incompatible with EF.

I had previously noted that the EF and the RF mounts were very similar in dimensions, but not compatible by design. As the Lens Kit doesn’t have any electrical or optical reasons why it can’t work, I wondered if it could be slightly modified to accept EF and RF.

Read More

Months after pledging on KickStarter, I finally received my TourBox! This device is meant to be used in your left hand while you work with a mouse or stylus with your right, hopefully speeding up your productivity in Adobe Photoshop, Lightroom, or any other app that accepts mouse and keyboard based shortcuts. After the initial setup, and about an hour of testing it out, I decided to disassemble it, as I do with most things I buy. Part of me wanted to inspect the workmanship of the innards of this device, as I’m sure there have been plenty of KickStarters with sloppy, quickly thrown together innards, where most people wouldn’t have a reason to look.

Read More

A friend of mine came to me with an interesting hardware mod request recently. He is very serious about security when it comes to cryptocurrency, and wanted to set up a totally air-gapped machine for doing key signing, among other things. For his hardware, he chose a Pinebook 64, as the hardware and OS are fully open sourced, therefore subject to more scrunchy by security-minded people. Because he wanted it to be air-gapped, he had initially gone the route of disabling the network adapters, but the fact that they were still physically only a few clicks away from being re-enabled, and internet-ready bothered him. So, we decided to physically disable the WiFi and Bluetooth completely, in order to be 100% sure that it stays offline.

Read More

The good folks over at Hak5 have created some really cool penetration testing hardware, my favorite being the USB Rubber Ducky. “The USB Rubber Ducky is a keystroke injection tool disguised as a generic flash drive.” It’s a very flexible, awesome tool in a small, unsuspecting package. I don’t actually own one of these devices, but to me, it fell under the category of “I bet it would be fun to build something like that for myself”.

Read More

Among other things, I run a Raspberry Pi as a print server. While clean install upgrading from Debian wheezy to squeeze, I found I was unable to get my Dell Laser Printer 1110 to work in CUPS. I had remembered it was a massive hassle last time, but I could not for the life of me remember what I did to make it work. Luckily, I backed up the whole SD card prior to wiping and reinstalling, so I was able to reference configuration and files. I’m sure there is a way to install these files using apt or by grabbing the sources somewhere… but I HATE dealing with peripherals drivers, so I cheated, copying the files from the old Pi the new Pi. Hopefully this helps someone else as as much as it will help my future-self.

Here are the three files required to get the printer working:

• /etc/cups/printers.conf
• /etc/cups/ppd/Dell_Laser_Printer_1110.ppd
• /usr/lib/cups/filter/rastertoqpdl

Update 2022-05-30: Fixed the dead links

I was in need of a way to feed large amounts JSON of data into Kinesis Firehose, so I could work out a bunch of other downstream logic. While the AWS CLI does allow me to push data, I have to call out each line I want to write as arguments to the command, and it has issues with some characters, meaning the data in didn’t always match the data out.

Though this seems like a pretty generic need, I couldn’t find any portable tools out there to push data from a file or pipe from stdin/stdout to a Firehose, so I decided to write my own, called pipe2firehose.

To install, download from Github release page or run:

1

go get github.com/Brayyy/pipe2firehose

Read More

As a software engineer for a company that generates analytics for advertisements, one technology that we leverage heavily is Node.js, which serves as a collection service for incoming status events when a user is watching a video. We had initially written the code that is doing this task a few years ago, when the newest stable version of Node.js was 0.10.18, and the “Express” module was in the 3.x branch. I’ve been looking to update our code to newer modules, and newer binaries, but updating doesn’t always mean a performance boost. I’ve read a lot about benchmarks comparing 0.10 and 0.12, and had seen people find 0.10 beating out the newer binaries, which fuels my caution about upgrading blindly into Node.js 4.1.0. Additionally, I have also been considering trying other web service modules, besides Express. Koa and Hapi have been getting attention, and I’ve seen benchmarking between all three, but I have yet to see a cross comparison with different versions of Node.js. I wanted to find out what branch of Node.js works the best with each module. Being that we have hundreds of Node.js processes behind load balancers, together handling 100s of million events per day, any minor change in performance could make a huge difference. If I’m going to make a change, I really want to be sure that it is a well informed one.

Read More

I picked up this relatively inexpensive Bluetooth device on Amazon recently, and while it functions very well, I was finding that the over-ear speakers let in too much air noise when on the freeway. So, I’m planning on replacing the speakers with some in-ear buds, or maybe just a female 1/8” audio jack, so I can switch headphones when I would like. As in any hack / mod, it’s best to start by doing your research. I could tear the current headset apart, but I wanted to see what I could learn first without ruining anything. Because I wasn’t able to find any photos of the internals of this device, so I have taken the photos myself, in hope that they help someone else.

Read More

I had been asked to recompile a VPAID ActionScript SWF with a minor change to the code, but I had a lot of stuff lingering in my build I would like to temporarily disable, by commenting out, then only commit the minor change asked by using prune, ie: git add -p file.as. There was a lot of things that had changed, and it was becoming difficult to spot the changes, with all of the commented out code everywhere.

Read More