Editing Preprocessor

{{short description|Program that processes input for another program}}
{{more citations needed|date=February 2013}}
{{Data transformation}}

In [[computer science]], a '''preprocessor''' (or '''precompiler''')<ref>{{cite web
   |url=https://www.ibm.com/docs/zosbasics/com.ibm.zos.zappldev/zappldev_111.htm
   |title=What is a precompiler?|website=[[IBM]] }}</ref> is a [[Computer program|program]] that processes its input data to produce output that is used as input in another program. The output is said to be a '''preprocessed''' form of the input data, which is often used by some subsequent programs like [[compiler]]s. The amount and kind of processing done depends on the nature of the preprocessor; some preprocessors are only capable of performing relatively simple textual substitutions and [[Macro (computer science)|macro]] expansions, while others have the power of full-fledged [[programming language]]s.

A common example from [[computer programming]] is the processing performed on [[source code]] before the next step of compilation.
In some [[computer language]]s (e.g., [[C (programming language)|C]] and [[PL/I (programming language)|PL/I]]) there is a phase of [[compiler|translation]] known as ''preprocessing''. It can also include macro processing, file inclusion and language extensions.

==Lexical preprocessors==
Lexical preprocessors are the lowest-level of preprocessors as they only require [[lexical analysis]], that is, they operate on the source text, prior to any [[parser|parsing]], by performing simple substitution of [[Lexical analysis|tokenized]] character sequences for other tokenized character sequences, according to user-defined rules. They typically perform [[macro substitution]], [[header file|textual inclusion]] of other files, and conditional compilation or inclusion.

===C preprocessor===
{{main|C preprocessor}}
The most common example of this is the [[C preprocessor]], which takes lines beginning with '#' as [[Directive (programming)|directives]].
The C preprocessor does not expect its input to use the syntax of the C language.
Some languages take a different approach and use built-in language features to achieve similar things. For example:
* Instead of macros, some languages use aggressive inlining and templates.
* Instead of includes, some languages use compile-time imports that rely on type information in the object code.
* Some languages use <code>if-then-else</code> and [[dead code elimination]] to achieve [[conditional compilation]].

===Other lexical preprocessors===
Other lexical preprocessors include the general-purpose [[m4 (language)|m4]], most commonly used in cross-platform build systems such as [[autoconf]], and [[general purpose macro processor|GEMA]], an open source macro processor which operates on patterns of context.

==Syntactic preprocessors==
Syntactic preprocessors were introduced with the [[Lisp programming language|Lisp]] family of languages. Their role is to transform syntax trees according to a number of user-defined rules. For some programming languages, the rules are written in the same language as the program (compile-time reflection). This is the case with [[Lisp programming language|Lisp]] and [[OCaml]]. Some other languages rely on a fully external language to define the transformations, such as the [[XSLT]] preprocessor for [[XML]], or its statically typed counterpart CDuce.

Syntactic preprocessors are typically used to customize the syntax of a language, extend a language by adding new primitives, or embed a [[domain-specific programming language]] (DSL) inside a general purpose language.

===Customizing syntax===
A good example of syntax customization is the existence of two different syntaxes in the [[OCaml|Objective Caml]] programming language.<ref>[http://caml.inria.fr/pub/docs/manual-camlp4/manual007.html The Revised syntax] from The Caml language website</ref> Programs may be written indifferently using the "normal syntax" or the "revised syntax", and may be pretty-printed with either syntax on demand.

Similarly, a number of programs written in [[OCaml]] customize the syntax of the language by the addition of new operators.

===Extending a language===
The best examples of language extension through macros are found in the [[Lisp programming language|Lisp]] family of languages. While the languages, by themselves, are simple dynamically typed functional cores, the standard distributions of [[Scheme (programming language)|Scheme]] or [[Common Lisp]] permit imperative or object-oriented programming, as well as static typing. Almost all of these features are implemented by syntactic preprocessing, although it bears noting that the "macro expansion" phase of compilation is handled by the compiler in Lisp. This can still be considered a form of preprocessing, since it takes place before other phases of compilation.

===Specializing a language===
One of the unusual features of the [[Lisp programming language|Lisp]] family of languages is the possibility of using macros to create an internal DSL. Typically, in a large [[Lisp programming language|Lisp]]-based project, a module may be written in a variety of such [[minilanguage]]s, one perhaps using a [[SQL]]-based dialect of [[Lisp programming language|Lisp]], another written in a dialect specialized for [[Graphical user interface|GUI]]s or pretty-printing, etc. [[Common Lisp]]'s standard library contains an example of this level of syntactic abstraction in the form of the LOOP macro, which implements an Algol-like minilanguage to describe complex iteration, while still enabling the use of standard Lisp operators.

The [[MetaOCaml]] preprocessor/language provides similar features for external DSLs. This preprocessor takes the description of the semantics of a language (i.e. an interpreter) and, by combining compile-time interpretation and code generation, turns that definition into a compiler to the [[OCaml]] programming language—and from that language, either to bytecode or to native code.
<!--
Please provide an authoritative cite defining "general purpose(sic) preprocessor" and
"general purpose template engine" or else remove this.

==Preprocessors as template engines==
Any "general purpose preprocessor", like [[M4 (computer language)|M4]], can be used as a [[template processor|template engine]].

"General purpose preprocessors" and "general purpose template engines" are the same kind of softwares.
--~~~
-->

==General purpose preprocessor==
{{main|General-purpose macro processor}}
Most preprocessors are specific to a particular data processing task (e.g., [[compilers|compiling]] the C language). A preprocessor may be promoted as being ''general purpose'', meaning that it is not aimed at a specific usage or programming language, and is intended to be used for a wide variety of text processing tasks.

{{see also|C preprocessor#Other uses}}
[[M4 (computer language)|M4]] is probably the most well known example of such a general purpose preprocessor, although the C preprocessor is sometimes used in a non-C specific role. Examples:
* using C preprocessor for [[JavaScript]] preprocessing.<ref>Show how to use [[C preprocessor|C-preprocessor]] on JavaScript files. [https://web.archive.org/web/20080116034428/http://blog.inetoffice.com/?p=12 "JavaScript is Not Industrial Strength"] ''by T. Snyder''.</ref><ref>{{cite web |title=The C Preprocessor in Javascript? |url=http://www.nongnu.org/espresso/js-cpp.html |website=espresso-mode}}</ref>
* using C preprocessor for [[devicetree]] processing within the [[Linux kernel]].<ref>{{Cite web|url=https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=b40b25fff8205dd18124d8fc87b2c9c57f269b5f|title=Kernel/Git/Stable/Linux.git - Linux kernel stable tree}}</ref>
* using [[M4 (computer language)|M4]] <small>(see on-article example)</small> or C preprocessor<ref>Show how to use [[C preprocessor|C-preprocessor]] as [[template processor|template engine]].
[https://jkorpela.fi/html/cpre.html "Using a C preprocessor as an HTML authoring tool"] ''by J. Korpela'', 2000.</ref> as a [[Web template system|template engine]], to [[HTML]] generation.
* [[imake]], a [[make (software)|make]] interface using the C preprocessor, written for the [[X Window System]] but now deprecated in favour of [[automake]].
* [[grompp]], a preprocessor for simulation input files for [[GROMACS]] (a fast, free, open-source code for some problems in [[computational chemistry]]) which calls the system C preprocessor (or other preprocessor as determined by the simulation input file) to parse the topology, using mostly the #define and #include mechanisms to determine the effective topology at grompp run time.

==See also==
* {{annotated link|Directive (programming)}}
* {{annotated link|Metaprogramming}}
* {{annotated link|Macro (computer science)|Macros}}
* {{annotated link|Source-to-source compiler|aka=Transpiler}}
* {{annotated link|Sass (stylesheet language)}}
* {{annotated link|Stylus (stylesheet language)}}
* {{annotated link|Less (stylesheet language)}}
* {{annotated link|Snippet (programming)}}
* {{annotated link|Template processor|Template engine}}
* The {{annotated link|C preprocessor}}
* The {{annotated link|camlp4|OCaml preprocessor-pretty-printer}}
* The {{annotated link|PL/I preprocessor}}
* The {{annotated link|Windows software trace preprocessor}}
* {{annotated link|General-purpose macro processor}}
<!-- open link
* The [[MetaOCaml]] metaprogramming language
* The [[Epigram (metaprogramming language)|Epigram]] metaprogramming language
* [[Oracle Pro*C]]
-->

==References==
{{reflist}}

==External links==
{{Wiktionary|preprocessor}}
* [https://web.archive.org/web/20090214092527/http://lispm.dyndns.org/news?ID=NEWS-2005-07-08-1 DSL Design in Lisp]
* [http://www.paulgraham.com/progbot.html Programming from the bottom up]
* The [https://logological.org/gpp Generic PreProcessor]
* Gema, the [http://gema.sourceforge.net General Purpose Macro Processor]
* The [[PIKT]] [http://pikt.org/pikt/ref/ref.4.piktc.html piktc] [http://pikt.org/pikt/ref/ref.3.html text, script, and configuration file preprocessor]
* [https://goetzpf.bitbucket.io/pyexpander pyexpander, a python based general purpose macro processor]
* [http://freshmeat.net/projects/minimac-macro-processor minimac, a minimalist macro processor]
* [https://github.com/raydac/java-comment-preprocessor Java Comment Preprocessor]

{{Authority control}}

[[Category:Programming language implementation]]