Inner-product spaces

 

Inner-product spaces[edit]

Main article: Inner product space

This section may require cleanup to meet Wikipedia's quality standards. The specific problem is: Need for a more encyclopedic style, which is homogeneous with the style of preceding sections. Also, some details do not belong to this general article but to more specialized ones. Also, inner product spaces should appear as a special instance of the more general concept of bilinear form. Finally, complex conjugation should appear in a specific section on linear algebra over the complexes. Please help improve this section if you can. (August 2018) (Learn how and when to remove this template message)

Besides these basic concepts, linear algebra also studies vector spaces with additional structure, such as an inner product. The inner product is an example of a bilinear form, and it gives the vector space a geometric structure by allowing for the definition of length and angles. Formally, an inner product is a map

${\displaystyle ⟨\cdot ,\cdot ⟩:�\times �\to �}$

that satisfies the following three axioms for all vectors u, v, w in V and all scalars a in F:^[19][20]

• Conjugate symmetry:

  ${\displaystyle ⟨\mathbf{�},\mathbf{�}⟩=\overline{⟨\mathbf{�},\mathbf{�}⟩}.}$

In ${\displaystyle \mathbb{�}}$, it is symmetric.

• Linearity in the first argument:

  

• Positive-definiteness:

  ${\displaystyle ⟨\mathbf{�},\mathbf{�}⟩\ge 0}$

with equality only for v = 0.

We can define the length of a vector v in V by

${\displaystyle \Vert \mathbf{�}{\Vert }^2=⟨\mathbf{�},\mathbf{�}⟩,}$

and we can prove the Cauchy–Schwarz inequality:

${\displaystyle |⟨\mathbf{�},\mathbf{�}⟩|\le \Vert \mathbf{�}\Vert \cdot \Vert \mathbf{�}\Vert .}$

In particular, the quantity

${\displaystyle \frac{|⟨\mathbf{�},\mathbf{�}⟩|}{\Vert \mathbf{�}\Vert \cdot \Vert \mathbf{�}\Vert }\le 1,}$

and so we can call this quantity the cosine of the angle between the two vectors.

Two vectors are orthogonal if ⟨u, v⟩ = 0. An orthonormal basis is a basis where all basis vectors have length 1 and are orthogonal to each other. Given any finite-dimensional vector space, an orthonormal basis could be found by the Gram–Schmidt procedure. Orthonormal bases are particularly easy to deal with, since if v = a₁ v₁ + ⋯ + a_n v_n, then

${\displaystyle {�}_{�}=⟨\mathbf{�},{\mathbf{�}}_{�}⟩.}$

The inner product facilitates the construction of many useful concepts. For instance, given a transform T, we can define its Hermitian conjugate T* as the linear transform satisfying

${\displaystyle ⟨�\mathbf{�},\mathbf{�}⟩=⟨\mathbf{�},{�}^{\ast }\mathbf{�}⟩.}$

If T satisfies TT* = T*T, we call T normal. It turns out that normal matrices are precisely the matrices that have an orthonormal system of eigenvectors that span V.

Relationship with geometry[edit]

There is a strong relationship between linear algebra and geometry, which started with the introduction by René Descartes, in 1637, of Cartesian coordinates. In this new (at that time) geometry, now called Cartesian geometry, points are represented by Cartesian coordinates, which are sequences of three real numbers (in the case of the usual three-dimensional space). The basic objects of geometry, which are lines and planes are represented by linear equations. Thus, computing intersections of lines and planes amounts to solving systems of linear equations. This was one of the main motivations for developing linear algebra.

Most geometric transformation, such as translations, rotations, reflections, rigid motions, isometries, and projections transform lines into lines. It follows that they can be defined, specified and studied in terms of linear maps. This is also the case of homographies and Möbius transformations, when considered as transformations of a projective space.

Until the end of the 19th century, geometric spaces were defined by axioms relating points, lines and planes (synthetic geometry). Around this date, it appeared that one may also define geometric spaces by constructions involving vector spaces (see, for example, Projective space and Affine space). It has been shown that the two approaches are essentially equivalent.^[21] In classical geometry, the involved vector spaces are vector spaces over the reals, but the constructions may be extended to vector spaces over any field, allowing considering geometry over arbitrary fields, including finite fields.

Presently, most textbooks, introduce geometric spaces from linear algebra, and geometry is often presented, at elementary level, as a subfield of linear algebra.

Usage and applications[edit]

Linear algebra is used in almost all areas of mathematics, thus making it relevant in almost all scientific domains that use mathematics. These applications may be divided into several wide categories.

Geometry of ambient space[edit]

The modeling of ambient space is based on geometry. Sciences concerned with this space use geometry widely. This is the case with mechanics and robotics, for describing rigid body dynamics; geodesy for describing Earth shape; perspectivity, computer vision, and computer graphics, for describing the relationship between a scene and its plane representation; and many other scientific domains.

In all these applications, synthetic geometry is often used for general descriptions and a qualitative approach, but for the study of explicit situations, one must compute with coordinates. This requires the heavy use of linear algebra.

Functional analysis[edit]

Functional analysis studies function spaces. These are vector spaces with additional structure, such as Hilbert spaces. Linear algebra is thus a fundamental part of functional analysis and its applications, which include, in particular, quantum mechanics (wave functions).

Study of complex systems[edit]

See also: Complex system

Most physical phenomena are modeled by partial differential equations. To solve them, one usually decomposes the space in which the solutions are searched into small, mutually interacting cells. For linear systems this interaction involves linear functions. For nonlinear systems, this interaction is often approximated by linear functions.^[b]This is called a linear model or first-order approximation. Linear models are frequently used for complex nonlinear real-world systems because it makes parametrization more manageable.^[22] In both cases, very large matrices are generally involved. Weather forecasting (or more specifically, parametrization for atmospheric modeling) is a typical example of a real-world application, where the whole Earth atmosphere is divided into cells of, say, 100km of width and 100km of height.

Scientific computation[edit]

Nearly all scientific computations involve linear algebra. Consequently, linear algebra algorithms have been highly optimized. BLAS and LAPACK are the best known implementations. For improving efficiency, some of them configure the algorithms automatically, at run time, for adapting them to the specificities of the computer (cache size, number of available cores, ...).

Some processors, typically graphics processing units (GPU), are designed with a matrix structure, for optimizing the operations of linear algebra.

Extensions and generalizations[edit]

This section presents several related topics that do not appear generally in elementary textbooks on linear algebra, but are commonly considered, in advanced mathematics, as parts of linear algebra.

Module theory[edit]

Main article: Module (mathematics)

The existence of multiplicative inverses in fields is not involved in the axioms defining a vector space. One may thus replace the field of scalars by a ring R, and this gives a structure called module over R, or R-module.

The concepts of linear independence, span, basis, and linear maps (also called module homomorphisms) are defined for modules exactly as for vector spaces, with the essential difference that, if R is not a field, there are modules that do not have any basis. The modules that have a basis are the free modules, and those that are spanned by a finite set are the finitely generated modules. Module homomorphisms between finitely generated free modules may be represented by matrices. The theory of matrices over a ring is similar to that of matrices over a field, except that determinants exist only if the ring is commutative, and that a square matrix over a commutative ring is invertible only if its determinant has a multiplicative inverse in the ring.

Vector spaces are completely characterized by their dimension (up to an isomorphism). In general, there is not such a complete classification for modules, even if one restricts oneself to finitely generated modules. However, every module is a cokernel of a homomorphism of free modules.

Modules over the integers can be identified with abelian groups, since the multiplication by an integer may identified to a repeated addition. Most of the theory of abelian groups may be extended to modules over a principal ideal domain. In particular, over a principal ideal domain, every submodule of a free module is free, and the fundamental theorem of finitely generated abelian groups may be extended straightforwardly to finitely generated modules over a principal ring.

There are many rings for which there are algorithms for solving linear equations and systems of linear equations. However, these algorithms have generally a computational complexity that is much higher than the similar algorithms over a field. For more details, see Linear equation over a ring.

Multilinear algebra and tensors[edit]

This section may require cleanup to meet Wikipedia's quality standards. The specific problem is: The dual space is considered above, and the section must be rewritten for given an understandable summary of this subject. Please help improve this section if you can. (September 2018) (Learn how and when to remove this template message)

In multilinear algebra, one considers multivariable linear transformations, that is, mappings that are linear in each of a number of different variables. This line of inquiry naturally leads to the idea of the dual space, the vector space V* consisting of linear maps f : V → F where F is the field of scalars. Multilinear maps T : Vⁿ → F can be described via tensor products of elements of V*.

If, in addition to vector addition and scalar multiplication, there is a bilinear vector product V × V → V, the vector space is called an algebra; for instance, associative algebras are algebras with an associate vector product (like the algebra of square matrices, or the algebra of polynomials).