O'Reilly logo

Understanding the Linux Kernel, 3rd Edition by Marco Cesati, Daniel P. Bovet

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Entering and Exiting a System Call

Native applications[*] can invoke a system call in two different ways:

  • By executing the int $0x80 assembly language instruction; in older versions of the Linux kernel, this was the only way to switch from User Mode to Kernel Mode.

  • By executing the sysenter assembly language instruction, introduced in the Intel Pentium II microprocessors; this instruction is now supported by the Linux 2.6 kernel.

Similarly, the kernel can exit from a system call—thus switching the CPU back to User Mode—in two ways:

  • By executing the iret assembly language instruction.

  • By executing the sysexit assembly language instruction, which was introduced in the Intel Pentium II microprocessors together with the sysenter instruction.

However, supporting two different ways to enter the kernel is not as simple as it might look, because:

  • The kernel must support both older libraries that only use the int $0x80 instruction and more recent ones that also use the sysenter instruction.

  • A standard library that makes use of the sysenter instruction must be able to cope with older kernels that support only the int $0x80 instruction.

  • The kernel and the standard library must be able to run both on older processors that do not include the sysenter instruction and on more recent ones that include it.

We will see in the section "Issuing a System Call via the sysenter Instruction" later in this chapter how the Linux kernel solves these compatibility problems.

Issuing a System Call via the int $0x80 ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required