Editing Database (section)

==History==
The sizes, capabilities, and performance of databases and their respective DBMSs have grown in orders of magnitude. These performance increases were enabled by the technology progress in the areas of [[Central processing unit|processors]], [[computer memory]], [[Computer data storage|computer storage]], and [[computer network]]s. The concept of a database was made possible by the emergence of direct access [[Data storage|storage media]] such as [[Hard disk drive|magnetic disks]], which became widely available in the mid-1960s; earlier systems relied on sequential storage of data on [[magnetic tape]]. The subsequent development of database technology can be divided into three eras based on data model or structure: [[navigational database|navigational]],{{sfn|Bachman|1973}} SQL/[[relational database|relational]], and post-relational.

The two main early navigational [[data model]]s were the [[Hierarchical database model|hierarchical model]] and the [[CODASYL]] model ([[network model]]). These were characterized by the use of [[Pointer (computer programming)|pointers]] (often physical disk addresses) to follow relationships from one record to another.

The [[relational model]], first proposed in 1970 by [[Edgar F. Codd]], departed from this tradition by insisting that [[Application software|applications]] should search for data by content, rather than by following links. The relational model employs sets of ledger-style tables, each used for a different type of [[entity]]. Only in the mid-1980s did computing hardware become powerful enough to allow the wide deployment of relational systems (DBMSs plus applications). By the early 1990s, however, relational systems dominated in all large-scale [[data processing]] applications, and {{as of | 2018 | lc = on}} they remain dominant: [[IBM Db2]], [[Oracle database|Oracle]], [[MySQL]], and [[Microsoft SQL Server]] are the most searched [[DBMS]].<ref>{{cite web|url=https://pypl.github.io/DB.html|title=TOPDB Top Database index|work=pypl.github.io}}</ref> The dominant database language, standardized SQL for the relational model, has influenced database languages for other data models.{{Citation needed|date=March 2013}}

[[Object database]]s were developed in the 1980s to overcome the inconvenience of [[object–relational impedance mismatch]], which led to the coining of the term "post-relational" and also the development of hybrid [[object–relational database]]s.

The next generation of post-relational databases in the late 2000s became known as [[NoSQL]] databases, introducing fast [[key–value store]]s and [[document-oriented database]]s. A competing "next generation" known as [[NewSQL]] databases attempted new implementations that retained the relational/SQL model while aiming to match the high performance of NoSQL compared to commercially available relational DBMSs.

===1960s, navigational DBMS===
{{Further|Navigational database}}
[[File:CodasylB.png|thumb|upright=1.15|Basic structure of navigational [[CODASYL]] database model]]

The introduction of the term ''database'' coincided with the availability of direct-access storage (disks and drums) from the mid-1960s onwards. The term represented a contrast with the tape-based systems of the past, allowing shared interactive use rather than daily [[batch processing]]. The [[Oxford English Dictionary]] cites a 1962 report by the [[System Development Corporation]] of California as the first to use the term "data-base" in a specific technical sense.<ref>{{cite web|title=database, n|url=http://www.oed.com/view/Entry/47411|work=OED Online|publisher=Oxford University Press|access-date=July 12, 2013|date=June 2013}} {{subscription required|s}}</ref>

As computers grew in speed and capability, a number of general-purpose database systems emerged; by the mid-1960s a number of such systems had come into commercial use. Interest in a standard began to grow, and [[Charles Bachman]], author of one such product, the [[Integrated Data Store]] (IDS), founded the [[Data Base Task Group|Database Task Group]] within [[CODASYL]], the group responsible for the creation and standardization of [[COBOL]]. In 1971, the Database Task Group delivered their standard, which generally became known as the ''CODASYL approach'', and soon a number of commercial products based on this approach entered the market.

The CODASYL approach offered applications the ability to navigate around a linked data set which was formed into a large network. Applications could find records by one of three methods:
#Use of a primary key (known as a CALC key, typically implemented by [[Hash function|hashing]])
#Navigating relationships (called ''sets'') from one record to another
#Scanning all the records in a sequential order

Later systems added [[B-tree]]s to provide alternate access paths. Many CODASYL databases also added a declarative query language for end users (as distinct from the navigational [[API]]). However, CODASYL databases were complex and required significant training and effort to produce useful applications.

[[IBM]] also had its own DBMS in 1966, known as [[IBM Information Management System|Information Management System]] (IMS). IMS was a development of software written for the [[Apollo program]] on the [[System/360]]. IMS was generally similar in concept to CODASYL, but used a strict hierarchy for its model of data navigation instead of CODASYL's network model. Both concepts later became known as navigational databases due to the way data was accessed: the term was popularized by Bachman's 1973 [[Turing Award]] presentation ''The Programmer as Navigator''. IMS is classified by IBM as a [[hierarchical database]]. IDMS and [[Cincom Systems]]' [[Cincom Systems#1970s|TOTAL]] databases are classified as network databases. IMS remains in use {{as of | 2014 | lc = on}}.<ref>{{cite web|last=IBM Corporation|title=IBM Information Management System (IMS) 13 Transaction and Database Servers delivers high performance and low total cost of ownership|url=http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=897&letternum=ENUS213-381|access-date=Feb 20, 2014|date=October 2013}}</ref>

===1970s, relational DBMS===
[[Edgar F. Codd]] worked at IBM in [[San Jose, California]], in an office primarily involved in the development of [[hard disk]] systems.{{r|rdbmsearlyyearsoh20070612}} He was unhappy with the navigational model of the CODASYL approach, notably the lack of a "search" facility. In 1970, he wrote a number of papers that outlined a new approach to database construction that eventually culminated in the groundbreaking ''A Relational Model of Data for Large Shared Data Banks''.{{sfn|Codd|1970}}

The paper described a new system for storing and working with large databases. Instead of records being stored in some sort of [[linked list]] of free-form records as in CODASYL, Codd's idea was to organize the data as a number of "[[Table (database)|tables]]", each table being used for a different type of entity. Each table would contain a fixed number of columns containing the attributes of the entity. One or more columns of each table were designated as a  [[primary key]] by which the rows of the table could be uniquely identified; cross-references between tables always used these primary keys, rather than disk addresses, and queries would join tables based on these key relationships, using a set of operations based on the mathematical system of [[relational calculus]] (from which the model takes its name). Splitting the data into a set of normalized tables (or ''relations'') aimed to ensure that each "fact" was only stored once, thus simplifying update operations. Virtual tables called ''views'' could present the data in different ways for different users, but views could not be directly updated.

Codd used mathematical terms to define the model: relations, tuples, and domains rather than tables, rows, and columns. The terminology that is now familiar came from early implementations. Codd would later criticize the tendency for practical implementations to depart from the mathematical foundations on which the model was based.

[[File:Relational key SVG.svg|thumb|In the [[relational model]], records are "linked" using virtual keys not stored in the database but defined as needed between the data contained in the records.]]
 
The use of primary keys (user-oriented identifiers) to represent cross-table relationships, rather than disk addresses, had two primary motivations. From an engineering perspective, it enabled tables to be relocated and resized without expensive database reorganization. But Codd was more interested in the difference in semantics: the use of explicit identifiers made it easier to define update operations with clean mathematical definitions, and it also enabled query operations to be defined in terms of the established discipline of [[first-order predicate calculus]]; because these operations have clean mathematical properties, it becomes possible to rewrite queries in provably correct ways, which is the basis of query optimization. There is no loss of expressiveness compared with the hierarchic or network models, though the connections between tables are no longer so explicit.

In the hierarchic and network models, records were allowed to have a complex internal structure. For example, the salary history of an employee might be represented as a "repeating group" within the employee record. In the relational model, the process of normalization led to such internal structures being replaced by data held in multiple tables, connected only by logical keys.

For instance, a common use of a database system is to track information about users, their name, login information, various addresses and phone numbers. In the navigational approach, all of this data would be placed in a single variable-length record. In the relational approach, the data would be ''normalized'' into a user table, an address table and a phone number table (for instance). Records would be created in these optional tables only if the address or phone numbers were actually provided.

As well as identifying rows/records using logical identifiers rather than disk addresses, Codd changed the way in which applications assembled data from multiple records. Rather than requiring applications to gather data one record at a time by navigating the links, they would use a declarative query language that expressed what data was required, rather than the access path by which it should be found. Finding an efficient access path to the data became the responsibility of the database management system, rather than the application programmer. This process, called query optimization, depended on the fact that queries were expressed in terms of mathematical logic.

Codd's paper inspired teams at various universities to research the subject, including one at [[University of California, Berkeley]]{{r|rdbmsearlyyearsoh20070612}} led by [[Eugene Wong]] and [[Michael Stonebraker]], who started [[INGRES]] using funding that had already been allocated for a geographical database project and student programmers to produce code. Beginning in 1973, INGRES delivered its first test products which were generally ready for widespread use in 1979. INGRES was similar to [[IBM System R|System R]] in a number of ways, including the use of a "language" for [[data access]], known as [[QUEL query languages|QUEL]]. Over time, INGRES moved to the emerging SQL standard.

IBM itself did one test implementation of the relational model, [[PRTV]], and a production one, [[IBM Business System 12|Business System 12]], both now discontinued. [[Honeywell]] wrote [[Multics Relational Data Store|MRDS]] for [[Multics]], and now there are two new implementations: [[Dataphor|Alphora Dataphor]] and Rel. Most other DBMS implementations usually called ''relational'' are actually SQL DBMSs.

In 1970, the University of Michigan began development of the [[MICRO Relational Database Management System|MICRO Information Management System]]{{sfn|Hershey|Easthope|1972}} based on [[David L. Childs|D.L. Childs]]' Set-Theoretic Data model.{{sfn|North|2010}}{{sfn|Childs|1968a}}{{sfn|Childs|1968b}} The university in 1974 hosted a debate between Codd and Bachman which Bruce Lindsay of IBM later described as "throwing lightning bolts at each other!".<ref name="rdbmsearlyyearsoh20070612">{{Cite interview |interviewer=Burton Grad |title=RDBMS Plenary 1: Early Years |url=https://archive.computerhistory.org/resources/access/text/2013/05/102702562-05-01-acc.pdf |access-date=2025-05-30 |publisher=Computer History Museum |date=2007-06-12}}</ref> MICRO was used to manage very large data sets by the [[US Department of Labor]], the [[U.S. Environmental Protection Agency]], and researchers from the [[University of Alberta]], the [[University of Michigan]], and [[Wayne State University]]. It ran on IBM mainframe computers using the [[Michigan Terminal System]].<ref name=MICROManual1977>{{cite book |author1=M.A. Kahn |author2=D.L. Rumelhart |author3=B.L. Bronson |date=October 1977 |url=https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B4t_NX-QeWDYZGMwOTRmOTItZTg2Zi00YmJkLTg4MTktN2E4MWU0YmZlMjE3 |title=MICRO Information Management System (Version 5.0) Reference Manual |publisher=Institute of Labor and Industrial Relations (ILIR), University of Michigan and Wayne State University}}</ref> The system remained in production until 1998.

===Integrated approach===
{{Main|Database machine}}

In the 1970s and 1980s, attempts were made to build database systems with integrated hardware and software. The underlying philosophy was that such integration would provide higher performance at a lower cost. Examples were [[IBM System/38]], the early offering of [[Teradata]], and the [[Britton Lee, Inc.]] database machine.

Another approach to hardware support for database management was [[International Computers Limited|ICL]]'s [[Content Addressable File Store|CAFS]] accelerator, a hardware disk controller with programmable search capabilities. In the long term, these efforts were generally unsuccessful because specialized database machines could not keep pace with the rapid development and progress of general-purpose computers. Thus most database systems nowadays are software systems running on general-purpose hardware, using general-purpose computer data storage. However, this idea is still pursued in certain applications by some companies like [[Netezza]] and Oracle ([[Exadata]]).

===Late 1970s, SQL DBMS===
IBM formed a team led by Codd that started working on a prototype system, ''[[IBM System R|System R]]'' despite opposition from others at the company.{{r|rdbmsearlyyearsoh20070612}} The first version was ready in 1974/5, and work then started on multi-table systems in which the data could be split so that all of the data for a record (some of which is optional) did not have to be stored in a single large "chunk". Subsequent multi-user versions were tested by customers in 1978 and 1979, by which time a standardized [[query language]] – SQL{{Citation needed|reason=First version of SQL standard was SQL-86 adopted in 1986|date=May 2012}} – had been added. Codd's ideas were establishing themselves as both workable and superior to CODASYL, pushing IBM to develop a true production version of System R, known as ''SQL/DS'', and, later, ''Database 2'' ([[IBM Db2]]).

[[Larry Ellison]]'s Oracle Database (or more simply, [[Oracle Database|Oracle]]) started from a different chain, based on IBM's papers on System R. Though Oracle V1 implementations were completed in 1978, it was not until Oracle Version 2 when Ellison beat IBM to market in 1979.<ref>{{cite web|url=https://www.oracle.com/us/corporate/profit/p27anniv-timeline-151918.pdf |archive-url=https://web.archive.org/web/20110320220813/http://www.oracle.com/us/corporate/profit/p27anniv-timeline-151918.pdf |archive-date=2011-03-20 |url-status=live |title=Oracle 30th Anniversary Timeline |access-date=23 August 2017}}</ref>

Stonebraker went on to apply the lessons from INGRES to develop a new database, Postgres, which is now known as [[PostgreSQL]]. PostgreSQL is often used for global mission-critical applications (the .org and .info domain name registries use it as their primary [[data store]], as do many large companies and financial institutions).

In Sweden, Codd's paper was also read and [[Mimer SQL]] was developed in the mid-1970s at [[Uppsala University]]. In 1984, this project was consolidated into an independent enterprise.

Another data model, the [[entity–relationship model]], emerged in 1976 and gained popularity for [[database design]] as it emphasized a more familiar description than the earlier relational model. Later on, entity–relationship constructs were retrofitted as a [[data modeling]] construct for the relational model, and the difference between the two has become irrelevant.{{Citation needed|date=March 2013}}

===1980s, on the desktop===
Besides IBM and various software companies such as [[Sybase]] and [[Informix Corporation]], most large computer hardware vendors by the 1980s had their own database systems such as [[DEC (company)|DEC]]'s [[VAX Rdb/VMS]].<ref name="rdbmslateryears20070612">{{Cite interview |interviewer=Burton Grad |title=RDBMS Plenary Session: The Later Years |url=https://archive.computerhistory.org/resources/access/text/2013/05/102701921-05-01-acc.pdf |access-date=2025-05-30 |publisher=Computer History Museum |date=2007-06-12}}</ref> The decade ushered in the age of [[Desktop Computer|desktop computing]]. The new computers empowered their users with spreadsheets like [[Lotus 1-2-3]] and database software like [[dBASE]]. The dBASE product was lightweight and easy for any computer user to understand out of the box. [[C. Wayne Ratliff]], the creator of dBASE, stated: "dBASE was different from programs like BASIC, C, FORTRAN, and COBOL in that a lot of the dirty work had already been done. The data manipulation is done by dBASE instead of by the user, so the user can concentrate on what he is doing, rather than having to mess with the dirty details of opening, reading, and closing files, and managing space allocation."<ref>[http://www.foxprohistory.org/interview_wayne_ratliff.htm Interview with Wayne Ratliff]. The FoxPro History. Retrieved on 2013-07-12.</ref> dBASE was one of the top selling software titles in the 1980s and early 1990s.

===1990s, object-oriented===
By the start of the decade databases had become a billion-dollar industry in about ten years.{{r|rdbmslateryears20070612}} The 1990s, along with a rise in [[object-oriented programming]], saw a growth in how data in various databases were handled. Programmers and designers began to treat the data in their databases as [[object (computer science)|objects]]. That is to say that if a person's data were in a database, that person's attributes, such as their address, phone number, and age, were now considered to belong to that person instead of being extraneous data. This allows for relations between data to be related to objects and their [[property (programming)|attributes]] and not to individual fields.<ref>Development of an object-oriented DBMS; Portland, Oregon, United States; Pages: 472–482; 1986; {{ISBN|0-89791-204-7}}</ref> The term "[[object–relational impedance mismatch]]" described the inconvenience of translating between programmed objects and database tables. [[Object database]]s and [[object–relational database]]s attempt to solve this problem by providing an object-oriented language (sometimes as extensions to SQL) that programmers can use as alternative to purely relational SQL. On the programming side, libraries known as [[object–relational mapping]]s (ORMs) attempt to solve the same problem.

===2000s, NoSQL and NewSQL===
{{Main|NoSQL|NewSQL}}

Database sales grew rapidly during the [[dotcom bubble]] and, after its end, the rise of [[ecommerce]]. The popularity of [[open source]] databases such as [[MySQL]] has grown since 2000, to the extent that Ken Jacobs of Oracle said in 2005 that perhaps "these guys are doing to us what we did to IBM".{{r|rdbmslateryears20070612}}

[[XML database]]s are a type of structured document-oriented database that allows querying based on [[XML]] document attributes. XML databases are mostly used in applications where the data is conveniently viewed as a collection of documents, with a structure that can vary from the very flexible to the highly rigid: examples include scientific articles, patents, tax filings, and personnel records.

[[NoSQL]] databases are often very fast{{Citation needed|date=May 2025}}, do not require fixed table schemas, avoid join operations by storing [[denormalization|denormalized]] data, and are designed to [[horizontal scaling|scale horizontally]].

In recent years, there has been a strong demand for massively distributed databases with high partition tolerance, but according to the [[CAP theorem]], it is impossible for a [[distributed computing|distributed system]] to simultaneously provide [[consistency model|consistency]], availability, and partition tolerance guarantees. A distributed system can satisfy any two of these guarantees at the same time, but not all three. For that reason, many NoSQL databases are using what is called [[eventual consistency]] to provide both availability and partition tolerance guarantees with a reduced level of data consistency.

[[NewSQL]] is a class of modern relational databases that aims to provide the same scalable performance of NoSQL systems for online transaction processing (read-write) workloads while still using SQL and maintaining the [[ACID (computer science)|ACID]] guarantees of a traditional database system.