How 90 lines of C saved the day
Setting up new remote servers is something we do all the time. It was a surprise, then, to get a call from our chief engineer one day, just after installing the operating system image for a new project. He had received a text message from our monitoring system (Nagios) to say that one of our other customer's live servers was no longer responding. We had re-imaged the wrong machine's disk!
The administration panel on our hosting company's website, identifies its servers only by long numbers, rather than names, which hadn't helped. But it was our responsibility to get our client's system back on line as soon as possible. We take complete backups of their system every night and store them off-site in encrypted files. However the key we needed to decrypt those backups was on the server we had just overwritten. Here's how we got it back:
After realising the mistake we split the RAID array on the server, in the hope that there would be salvageable data on the drive (the disk had not been zeroed during the re-imaging process). Then we tried to use the open source recovery tool photorec to get back the files. This didn't work out well: it returned gigabytes of false positives which were not easy to search through.
We knew that the key began with the bytes 0x95 and 0x01, and was smaller than 4kB. So we wrote a C program to search for these bytes among the raw data on the disk, and write out the following 4000 bytes, back-tracking the reads each time in case of repeats. After a few tests, to convince ourselves the program was working, we set it running, and went home.
When we checked later, the program had finished and had created thousands of files with possible matches. An unencrypted key contains an email address, so all we had to do was search these files for @aptivate.org. There were only three matches, and the first of these turned out to be the missing key. We were lucky that installing a new operating system had not overwritten this part of the disc. Using the key, we were able to decrypt and restore the backup data and tell our client the good news.
Here is the source code for the key recovery program. It was written as a quick hack for this one-off operation, and is not particularly elegant: there is an unusual mix of static and dynamic memory allocation, and the memcpy would fail if BUFFSIZE was closer to the length of the files written. But it did what we wanted, and got us out of a hole.
#include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/types.h> #include <dirent.h> #include <fcntl.h> #include <unistd.h> #include <sys/stat.h>#define BUFFSIZE 1024*1000
int main(int argc, char **argv) { int fd = open(argv[1], O_RDONLY); int i, c = 0; int file_counter = 0;
char * buffer; buffer = malloc(BUFFSIZE); char dirname[1000];
int offset = 0; int dir_counter = 0;
while (read(fd, buffer+offset, BUFFSIZE-offset) != 0) { int save;
for (i = 0; i < BUFFSIZE; i++) { FILE *fout;
switch (c) { case 0: if (buffer[i] == (char) 0x95) c++; break; case 1: if (buffer[i] == (char) 0x01) { if ((file_counter % 10000) == 0) { sprintf(dirname, "d%.8d", dir_counter); mkdir(dirname, 0740); printf("%s\n", dirname); dir_counter++; } char * filename = malloc(1000); sprintf(filename, "./%s/key%.8d", dirname, file_counter); printf("%s\n", filename); file_counter++; fout = fopen(filename, "wb"); free(filename); fputc(0x95, fout); fputc(0x01, fout); c++; save = i; printf("%x\n", i); } else c = 0; break; default: if (c <= 4000) { fputc(buffer[i], fout); c++; } else { fclose(fout); c = 0; i = save; } break;
} }
if (c > 1) { memcpy(buffer, buffer+save, BUFFSIZE - i); offset = BUFFSIZE - i; save = 0; } }
close(fd); return 0; }