What is a magic file?

This is a file used by the file utility program in UNIX-based operating systems to identify a file type by examining its content rather than relying on its file extension or metadata.

Structure of a magic file

Magic files contain plain text content that typically follows a specific format defined as rules on each line with the second line specifying the mime type

Each rule consists of four fields separated by tabs or spaces as shown

offset: This field specifies where in the file to start looking for the pattern. It can be an absolute value, for example, 0 (meaning from the beginning of the file), 10 (meaning skip 10 bytes from the beginning of the file and start looking at the 11th byte in the file), It can also be a symbolic reference to another offset in the file.

type: This field specifies how the data at the offset should be interpreted. Commonly used types include string, long, byte, short, and search.

test: This field describes the precise pattern or tests that should be run on the data at the offset. This can be a literal string, a regular expression, or a more complex test.

description: An explanation of the file type in human-readable form is provided in this field. It is usually a short phrase describing.

!:mime: This is used as a directive to specify the mime type of the file.

Let's create a magic file

Begin by creating two files. file1 and file2 with some content

Let's now create a magic file named magic that detects "FOO" in a file using our desired text editor, in this case, the vi editor.

Shown in the image is the content of the magic file, with 0 offset typestring, test FOO, and a custom mime typetext/foo.

Testing the magic file using the file command with -m magic_file_name in this casemagic and the name of the file to test. Passing the --mime-type option outputs the mime type of the file

We can compile our magic file to a binary by passing the -C option to the file command. This comes with some pros,

Improved performance. Once compiled to binary, it will be loaded faster into memory and executed very quickly.

Portability. It can be easily distributed onto other systems without requiring original magic files once it is compiled

The command to compile the magic file is file -C -m magic_filename the output magic_filename.mgc

In this case, our command will be file -C -m magic it will generate the magic.mgc file

Refer to the Linux manual page of the file command and magic manual to get more information about them. This will enable you even generate more complex patterns in magic files

Thank you😊