How to compile assembly code with NASM: in Linux, Windows

An NASM assembler will turn your low-level coding, using mnemonics, into machine language that can be understood by the processor. This article will not teach you to program with NASM, but to create an executable command for Linux and Windows from NASM source code. We will also give some guidance on how to use NASM under macOS.

Netwide Assembler (NASM) is an assembler and dissembler for the Intel x86 architecture and is commonly used to create 16-bit, 32-bit (IA-32), and 64-bit (x86-64) programs. 

How to compile an assembly program with NASM for Linux?

Creating the source file

• You can use any text editor, such as Gedit, KWrite, or XEmacs, to do so. When you save your file, give it the extension .asm.

Assembling the source file

• For this step, you will need NASM software installed on your machine.
• If you're running Debian or Ubuntu, simply type the command:

sudo apt-get install nasm

• If you have another Linux distribution, you must use your distribution's package manager (e.g. Urpmi, Yum, Emerge) or download NASM from the official site.
• Use the following command line to assemble your source file:

nasm -f elf test.asm

• In the example, the saved .asm file is called test.asm. This will create a file named test.o in the current directory.

N.B. This file is not executable. It is still an object file.

Creating the executable

• Now that we have our object file, named test.o, we must create our executable.
• Your program may begin with a procedure called _start. This means that your program has its own point of entry, without the use of the main function. However, you'll need to use the "l" to create your executable:

ld test.o -o test

• Alternatively, your program may begin with a procedure called main. You will need to use gcc to create your executable:

gcc test.o -o test

• Now, your executable is created, tested, and located in the current directory.

Program execution

• To run the program called test, just type this command:

. / test 

How to compile an assembly program with NASM for Windows?

• The main function is not available under Windows and must be replaced by WinMain.
• If your entry point is _start or main, it should be changed to _WinMain @ 16. Also, change the ret at the end of the procedure to ret 16:

section .text       
global _WinMain@16       
_WinMain@16:       
 mov eax, 0       
 ret 16 

Installing the software

• You must first install NASM. Keep an archive somewhere, as it will be used later.

• The most difficult step will be installing MinGW, which is a free development environment for Windows:
• Start by choosing the latest version of MingGW from their website. Run the installer, but don't update at this point. Leave all options selected by default, and wait for it to install.
• Now you need to insert NASM in the development environment MinGW. Unpack the NASM archive. You should get a folder containing, among other things, a file named nasm.exe. Copy this file into the directory C: \ MinGW \ bin.

Creating a source file

• Like Linux, there is no need to use a specific publisher to create a source file for NASM. You can use Notepad. But take note that it tends to add the .txt extension to files it creates. To remove any ambiguity, it is recommend that you view the extensions of your files.
• In any event, avoid word processors, such as Word or WordPad.
• If you wish, you can also use an editor that uses NASM syntax, such a NasmEdit IDE.
• Make sure your save your source file with the .asm extension.

Assembling the source file

• Open the Command window by going to Start > Run and typing cmd.exe
• Using the command cd, go to the folder containing your source file. Once you are in this directory, assemble your source file (test.asm) with this command:

nasm -f win32 test.asm -o test.o

• You have now created an object file. The next step will be to turn it into an executable file.

Creation and execution of the program

• From your Command window, type the final command to create the executable:

ld test.o -o test.exe 

Can I compile an assembly program with NASM on MacOS?

If you try to use NASM on a Mac using standard Linux or Windows guides, you will fail. However, there are tricks with which you can work with NASM under 64-bit macOS. It is important to consider the following differences and the fact that slightly different settings are required here:

• Under MacOS, completely different system call numbers operate.
• The object file format under MacOS is macho64 (not elf64).
• Characters shared by modules start with an underscore prefix.
• You should add rel when you refer to marked memory areas, and add lea to get your addresses. This is because macOS does not allow absolute addressing.

You can find more detailed tips for using MacOS in the NASM Guide. Also use Homebrew to install NASM on Apple computers.