Study Guides/Computer Science/Tokens in C Language
Study Guide ยท Computer Science

What are Tokens in C Language?

When you write an English sentence, you use words, punctuation marks, and spaces. Similarly, when you write a program in the C language, you use specific basic elements. The compiler breaks down your entire program into the smallest individual units, which are called Tokens.

Question (Click to Flip)

Are spaces considered tokens?

Answer

No. Blank spaces, tabs, and newlines are known as 'white spaces'. The compiler uses them to separate tokens from each other, but the spaces themselves are ignored during compilation.

Card 1 of 1 free previews

Key Facts

There are exactly 32 standard keywords defined in the original ANSI C programming language.

Definition

A Token is the smallest individual unit in a C program. It is the basic building block of the language. The C compiler cannot break down a token any further.

The 6 Types of C Tokens

Every single character or word you type in a C program falls into one of these six categories:

  1. Keywords: Pre-defined, reserved words that have a special meaning to the compiler. (e.g., int, return, if, while). You cannot use them as variable names.
  2. Identifiers: Names created by the programmer for variables, functions, or arrays. (e.g., main, age, total_sum).
  3. Constants: Fixed values that do not change during the execution of a program. (e.g., 10, 3.14, 'A').
  4. Strings: A sequence of characters enclosed inside double quotes. (e.g., "Hello World").
  5. Special Symbols: Punctuation marks and brackets that have specific grammatical uses in C. (e.g., [ ], { }, ,, ;, #).
  6. Operators: Symbols that trigger mathematical or logical actions. (e.g., +, -, =, &&).

Example Breakdown

Look at this simple line of code: int sum = 10; The compiler breaks it down into 5 tokens:

  1. int (Keyword)
  2. sum (Identifier)
  3. = (Operator)
  4. 10 (Constant)
  5. ; (Special Symbol)

Questions and Answers

Are spaces considered tokens?+

No. Blank spaces, tabs, and newlines are known as 'white spaces'. The compiler uses them to separate tokens from each other, but the spaces themselves are ignored during compilation.

More in Computer Science

Study Smarter with Shinyu.ai

Turn this guide into revision flashcards, a practice exam, or an AI-generated podcast โ€” free, no signup required.