Visión por computador

Computerized information extraction from images

La visión por computadora es un campo científico interdisciplinario que se ocupa de cómo las computadoras pueden obtener una comprensión de alto nivel a partir de imágenes o videos digitales . Desde la perspectiva de la ingeniería , busca comprender y automatizar tareas que el sistema visual humano puede realizar. ^{[1] [2] [3]}

Las tareas de visión por computadora incluyen métodos para adquirir , procesar , analizar y comprender imágenes digitales, y extraer datos de alta dimensión del mundo real para producir información numérica o simbólica, por ejemplo, en forma de decisiones. ^{[4] [5] [6] [7] La} comprensión en este contexto significa la transformación de imágenes visuales (la entrada de la retina) en descripciones del mundo que tienen sentido para los procesos de pensamiento y pueden provocar la acción apropiada. Esta comprensión de la imagen puede verse como la separación de la información simbólica de los datos de la imagen utilizando modelos construidos con la ayuda de la geometría, la física, la estadística y la teoría del aprendizaje. ^[8]

La disciplina científica de la visión por computadora se ocupa de la teoría detrás de los sistemas artificiales que extraen información de las imágenes. Los datos de imagen pueden tomar muchas formas, como secuencias de video, vistas desde múltiples cámaras, datos multidimensionales de un escáner 3D o un dispositivo de escaneo médico. La disciplina tecnológica de la visión por computadora busca aplicar sus teorías y modelos a la construcción de sistemas de visión por computadora.

Los subdominios de la visión por computadora incluyen reconstrucción de escenas , detección de objetos , detección de eventos, seguimiento de video , reconocimiento de objetos , estimación de pose en 3D , aprendizaje, indexación, estimación de movimiento , servo visual , modelado de escenas en 3D y restauración de imágenes . ^[6]

Definición

La visión por computadora es un campo interdisciplinario que se ocupa de cómo se pueden hacer las computadoras para obtener una comprensión de alto nivel a partir de imágenes o videos digitales . Desde la perspectiva de la ingeniería , busca automatizar tareas que el sistema visual humano puede realizar. ^{[1] [2] [3]} "La visión por computadora se ocupa de la extracción, el análisis y la comprensión automáticos de información útil a partir de una sola imagen o una secuencia de imágenes. Implica el desarrollo de una base teórica y algorítmica para lograr la comprensión visual automática . " ^[9] Como disciplina científica, la visión por computadora se ocupa de la teoría detrás de los sistemas artificiales que extraen información de las imágenes. Los datos de imagen pueden tomar muchas formas, como secuencias de video, vistas desde múltiples cámaras o datos multidimensionales de un escáner médico . ^[10] Como disciplina tecnológica, la visión por computadora busca aplicar sus teorías y modelos para la construcción de sistemas de visión por computadora.

Historia

A fines de la década de 1960, la visión por computadora comenzó en universidades que eran pioneras en inteligencia artificial . Estaba destinado a imitar el sistema visual humano , como un trampolín para dotar a los robots de un comportamiento inteligente. ^[11] En 1966, se creía que esto podría lograrse a través de un proyecto de verano, conectando una cámara a una computadora y haciendo que "describiera lo que veía". ^{[12] [13]}

Lo que distinguía la visión por computadora del campo predominante del procesamiento de imágenes digitales en ese momento era el deseo de extraer una estructura tridimensional de las imágenes con el objetivo de lograr una comprensión completa de la escena. Los estudios realizados en la década de 1970 formaron los primeros cimientos de muchos de los algoritmos de visión por computadora que existen en la actualidad, incluida la extracción de bordes de imágenes, etiquetado de líneas, modelado no poliédrico y poliédrico , representación de objetos como interconexiones de estructuras más pequeñas, flujo óptico y estimación de movimiento . ^[11]

La siguiente década vio estudios basados ​​en análisis matemáticos más rigurosos y aspectos cuantitativos de la visión por computadora. Estos incluyen el concepto de espacio de escala , la inferencia de la forma a partir de varias señales como el sombreado , la textura y el enfoque, y los modelos de contorno conocidos como serpientes . Los investigadores también se dieron cuenta de que muchos de estos conceptos matemáticos podrían tratarse dentro del mismo marco de optimización que la regularización y los campos aleatorios de Markov . ^[14] En la década de 1990, algunos de los temas de investigación anteriores se volvieron más activos que otros. La investigación en reconstrucciones proyectivas en 3-D permitió comprender mejor la calibración de la cámara. Con el advenimiento de los métodos de optimización para la calibración de la cámara, se advirtió que muchas de las ideas ya se habían explorado en la teoría del ajuste de paquetes del campo de la fotogrametría . Esto condujo a métodos para reconstrucciones tridimensionales escasas de escenas a partir de múltiples imágenes . Se avanzó en el problema de la correspondencia estéreo densa y en otras técnicas estéreo de múltiples vistas. Al mismo tiempo, se utilizaron variaciones del corte del gráfico para resolver la segmentación de la imagen . Esta década también marcó la primera vez que se utilizaron en la práctica técnicas de aprendizaje estadístico para reconocer rostros en imágenes (ver Eigenface). Hacia fines de la década de 1990, se produjo un cambio significativo con la mayor interacción entre los campos de los gráficos por computadora y la visión por computadora. Esto incluyó renderizado basado en imágenes , transformación de imágenes , interpolación de vistas , unión de imágenes panorámicas y renderizado de campo de luz temprano . ^[11]

El trabajo reciente ha visto el resurgimiento de métodos basados ​​en funciones , utilizados junto con técnicas de aprendizaje automático y marcos de optimización complejos. ^{[15] [16]} El avance de las técnicas de aprendizaje profundo ha dado más vida al campo de la visión por computadora. La precisión de los algoritmos de aprendizaje profundo en varios conjuntos de datos de visión por computadora de referencia para tareas que van desde la clasificación, la segmentación y el flujo óptico ha superado los métodos anteriores. ^{[ cita requerida ]}

Campos relacionados

Detección de objetos en una fotografía

Física del estado sólido

La física del estado sólido es otro campo que está estrechamente relacionado con la visión por computadora. La mayoría de los sistemas de visión por computadora se basan en sensores de imagen , que detectan la radiación electromagnética , que generalmente se presenta en forma de luz visible o infrarroja . Los sensores están diseñados utilizando física cuántica . El proceso por el cual la luz interactúa con las superficies se explica mediante la física. La física explica el comportamiento de la óptica, que es una parte fundamental de la mayoría de los sistemas de imágenes. Los sensores de imagen sofisticados incluso requieren la mecánica cuántica para proporcionar una comprensión completa del proceso de formación de imágenes. ^[11] Además, se pueden abordar varios problemas de medición en física utilizando visión por computadora, por ejemplo, movimiento en fluidos.

Neurobiología

Un tercer campo que juega un papel importante es la neurobiología , concretamente el estudio del sistema de visión biológica. Durante el último siglo, se ha realizado un extenso estudio de los ojos, las neuronas y las estructuras cerebrales dedicadas al procesamiento de los estímulos visuales tanto en humanos como en varios animales. Esto ha llevado a una descripción burda, aunque complicada, de cómo funcionan los sistemas de visión "reales" para resolver determinadas tareas relacionadas con la visión. Estos resultados han llevado a un subcampo dentro de la visión por computadora donde los sistemas artificiales están diseñados para imitar el procesamiento y el comportamiento de los sistemas biológicos, en diferentes niveles de complejidad. Además, algunos de los métodos basados ​​en el aprendizaje desarrollados dentro de la visión por computadora ( por ejemplo, redes neuronales y aprendizaje profundo análisis y clasificación de imágenes y características) tienen su experiencia en biología.

Algunas ramas de la investigación de la visión por computadora están estrechamente relacionadas con el estudio de la visión biológica ; de hecho, al igual que muchas ramas de la investigación de la IA están estrechamente relacionadas con la investigación sobre la conciencia humana y el uso del conocimiento almacenado para interpretar, integrar y utilizar información visual. El campo de la visión biológica estudia y modela los procesos fisiológicos detrás de la percepción visual en humanos y otros animales. La visión por computadora, por otro lado, estudia y describe los procesos implementados en software y hardware detrás de los sistemas de visión artificial. El intercambio interdisciplinario entre visión biológica e informática ha resultado fructífero para ambos campos. ^[17]

Procesamiento de señales

Otro campo relacionado con la visión por computadora es el procesamiento de señales . Muchos métodos para el procesamiento de señales de una variable, típicamente señales temporales, se pueden extender de forma natural al procesamiento de señales de dos variables o señales de múltiples variables en la visión por computadora. Sin embargo, debido a la naturaleza específica de las imágenes, existen muchos métodos desarrollados dentro de la visión por computadora que no tienen contrapartida en el procesamiento de señales de una variable. Junto con la multidimensionalidad de la señal, esto define un subcampo en el procesamiento de señales como parte de la visión por computadora.

Navegación robótica

La navegación con robots a veces se ocupa de la planificación o deliberación autónoma de rutas para que los sistemas robóticos naveguen por un entorno . ^[18] Se requiere una comprensión detallada de estos entornos para navegar a través de ellos. La información sobre el medio ambiente podría ser proporcionada por un sistema de visión por computadora, actuando como un sensor de visión y proporcionando información de alto nivel sobre el medio ambiente y el robot.

Otros campos

Además de los puntos de vista antes mencionados sobre la visión por computadora, muchos de los temas de investigación relacionados también pueden estudiarse desde un punto de vista puramente matemático. Por ejemplo, muchos métodos en visión artificial se basan en estadísticas , optimización o geometría . Finalmente, una parte significativa del campo se dedica al aspecto de implementación de la visión por computadora; cómo se pueden realizar los métodos existentes en varias combinaciones de software y hardware, o cómo se pueden modificar estos métodos para ganar velocidad de procesamiento sin perder demasiado rendimiento. La visión por computadora también se utiliza en el comercio electrónico de moda, la gestión de inventarios, la búsqueda de patentes, los muebles y la industria de la belleza. ^{[ cita requerida ]}

Distinciones

Los campos más estrechamente relacionados con la visión por computadora son el procesamiento de imágenes , el análisis de imágenes y la visión artificial . Existe una superposición significativa en la gama de técnicas y aplicaciones que cubren. Esto implica que las técnicas básicas que se utilizan y desarrollan en estos campos son similares, algo que se puede interpretar ya que solo hay un campo con diferentes nombres. Por otro lado, parece necesario que los grupos de investigación, revistas científicas, congresos y empresas se presenten o comercialicen como pertenecientes específicamente a uno de estos campos y, por tanto, se han realizado diversas caracterizaciones que distinguen a cada uno de los campos de los demás. presentado.

Los gráficos por computadora producen datos de imágenes a partir de modelos 3D, la visión por computadora a menudo produce modelos en 3D a partir de datos de imágenes. ^[19] También existe una tendencia hacia una combinación de las dos disciplinas, por ejemplo , como se explora en la realidad aumentada .

Las siguientes caracterizaciones parecen relevantes pero no deben tomarse como universalmente aceptadas:

• El procesamiento de imágenes y el análisis de imágenes tienden a enfocarse en imágenes 2D, cómo transformar una imagen en otra, por ejemplo , mediante operaciones de píxeles como mejora de contraste, operaciones locales como extracción de bordes o eliminación de ruido, o transformaciones geométricas como rotar la imagen. . Esta caracterización implica que el procesamiento / análisis de imágenes no requiere suposiciones ni produce interpretaciones sobre el contenido de la imagen.
• La visión por computadora incluye análisis 3D a partir de imágenes 2D. Analiza la escena 3D proyectada en una o varias imágenes, por ejemplo , cómo reconstruir la estructura u otra información sobre la escena 3D a partir de una o varias imágenes. La visión por computadora a menudo se basa en suposiciones más o menos complejas sobre la escena representada en una imagen.
• La visión artificial es el proceso de aplicar una variedad de tecnologías y métodos para proporcionar inspección automática basada en imágenes, control de procesos y guía de robots ^[20] en aplicaciones industriales. ^[17] La visión artificial tiende a centrarse en aplicaciones, principalmente en la fabricación, por ejemplo , robots y sistemas basados ​​en la visión para la inspección, la medición o el picking basados ​​en la visión (como el picking en contenedores ^[21]). This implies that image sensor technologies and control theory often are integrated with the processing of image data to control a robot and that real-time processing is emphasised by means of efficient implementations in hardware and software. It also implies that the external conditions such as lighting can be and are often more controlled in machine vision than they are in general computer vision, which can enable the use of different algorithms.
• There is also a field called imaging which primarily focuses on the process of producing images, but sometimes also deals with processing and analysis of images. For example, medical imaging includes substantial work on the analysis of image data in medical applications.
• Finally, pattern recognition is a field which uses various methods to extract information from signals in general, mainly based on statistical approaches and artificial neural networks.^[22] A significant part of this field is devoted to applying these methods to image data.

Photogrammetry also overlaps with computer vision, e.g., stereophotogrammetry vs. computer stereo vision.

Applications

Applications range from tasks such as industrial machine vision systems which, say, inspect bottles speeding by on a production line, to research into artificial intelligence and computers or robots that can comprehend the world around them. The computer vision and machine vision fields have significant overlap. Computer vision covers the core technology of automated image analysis which is used in many fields. Machine vision usually refers to a process of combining automated image analysis with other methods and technologies to provide automated inspection and robot guidance in industrial applications. In many computer-vision applications, the computers are pre-programmed to solve a particular task, but methods based on learning are now becoming increasingly common. Examples of applications of computer vision include systems for:

Learning 3D shapes has been a challenging task in computer vision. Recent advances in deep learning has enabled researchers to build models that are able to generate and reconstruct 3D shapes from single or multi-view depth maps or silhouettes seamlessly and efficiently ^[19]

• Automatic inspection, e.g., in manufacturing applications;
• Assisting humans in identification tasks, e.g., a species identification system;^[23]
• Controlling processes, e.g., an industrial robot;
• Detecting events, e.g., for visual surveillance or people counting, e.g., in the restaurant industry;
• Interaction, e.g., as the input to a device for computer-human interaction;
• Modeling objects or environments, e.g., medical image analysis or topographical modeling;
• Navigation, e.g., by an autonomous vehicle or mobile robot; and
• Organizing information, e.g., for indexing databases of images and image sequences.

Medicine

Play media

DARPA's Visual Media Reasoning concept video

One of the most prominent application fields is medical computer vision, or medical image processing, characterized by the extraction of information from image data to diagnose a patient. An example of this is detection of tumours, arteriosclerosis or other malign changes; measurements of organ dimensions, blood flow, etc. are another example. It also supports medical research by providing new information: e.g., about the structure of the brain, or about the quality of medical treatments. Applications of computer vision in the medical area also includes enhancement of images interpreted by humans—ultrasonic images or X-ray images for example—to reduce the influence of noise.

Machine Vision

A second application area in computer vision is in industry, sometimes called machine vision, where information is extracted for the purpose of supporting a manufacturing process. One example is quality control where details or final products are being automatically inspected in order to find defects. Another example is measurement of position and orientation of details to be picked up by a robot arm. Machine vision is also heavily used in agricultural process to remove undesirable food stuff from bulk material, a process called optical sorting.^[24]

Military

Military applications are probably one of the largest areas for computer vision. The obvious examples are detection of enemy soldiers or vehicles and missile guidance. More advanced systems for missile guidance send the missile to an area rather than a specific target, and target selection is made when the missile reaches the area based on locally acquired image data. Modern military concepts, such as "battlefield awareness", imply that various sensors, including image sensors, provide a rich set of information about a combat scene which can be used to support strategic decisions. In this case, automatic processing of the data is used to reduce complexity and to fuse information from multiple sensors to increase reliability.

Autonomous vehicles

Artist's concept of Curiosity, an example of an uncrewed land-based vehicle. Notice the stereo camera mounted on top of the rover.

One of the newer application areas is autonomous vehicles, which include submersibles, land-based vehicles (small robots with wheels, cars or trucks), aerial vehicles, and unmanned aerial vehicles (UAV). The level of autonomy ranges from fully autonomous (unmanned) vehicles to vehicles where computer-vision-based systems support a driver or a pilot in various situations. Fully autonomous vehicles typically use computer vision for navigation, e.g. for knowing where it is, or for producing a map of its environment (SLAM) and for detecting obstacles. It can also be used for detecting certain task-specific events, e.g., a UAV looking for forest fires. Examples of supporting systems are obstacle warning systems in cars and systems for autonomous landing of aircraft. Several car manufacturers have demonstrated systems for autonomous driving of cars, but this technology has still not reached a level where it can be put on the market. There are ample examples of military autonomous vehicles ranging from advanced missiles to UAVs for recon missions or missile guidance. Space exploration is already being made with autonomous vehicles using computer vision, e.g., NASA's Curiosity and CNSA's Yutu-2 rover.

Tactile Feedback

Rubber artificial skin layer with flexible structure for shape estimation of micro-undulation surfaces

Above is a silicon mold with a camera inside containing many different point markers. When this sensor is pressed against the surface the silicon deforms and the position of the point markers shift. A computer can then take this data and determine how exactly the mold is pressed against the surface. This can be used to calibrate robotic hands in order to make sure they can grasp objects effectively.

Materials such as rubber and silicon are being used to create sensors that allow for applications such as detecting micro undulations and calibrating robotic hands. Rubber can be used in order to create a mold that can be placed over a finger, inside of this mold would be multiple strain gauges. The finger mold and sensors could then be placed on top of a small sheet of rubber containing an array of rubber pins. A user can then wear the finger mold and trace a surface. A computer can then read the data from the strain gauges and measure if one or more of the pins is being pushed upward. If a pin is being pushed upward then the computer can recognize this as an imperfection in the surface. This sort of technology is useful in order to receive accurate data of the imperfections on a very large surface.^[25] Another variation of this finger mold sensor are sensors that contain a camera suspended in silicon. The silicon forms a dome around the outside of the camera and embedded in the silicon are point markers that are equally spaced. These cameras can then be placed on devices such as robotic hands in order to allow the computer to receive highly accurate tactile data.^[26]

Other application areas include:

• Support of visual effects creation for cinema and broadcast, e.g., camera tracking (matchmoving).
• Surveillance.
• Driver drowsiness detection^{[citation needed]}
• Tracking and counting organisms in the biological sciences^[27]

Typical tasks

Each of the application areas described above employ a range of computer vision tasks; more or less well-defined measurement problems or processing problems, which can be solved using a variety of methods. Some examples of typical computer vision tasks are presented below.

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions.^[4][5][6][7] Understanding in this context means the transformation of visual images (the input of the retina) into descriptions of the world that can interface with other thought processes and elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.^[8]

Recognition

The classical problem in computer vision, image processing, and machine vision is that of determining whether or not the image data contains some specific object, feature, or activity. Different varieties of the recognition problem are described in the literature:^{[citation needed]}

• Object recognition (also called object classification) – one or several pre-specified or learned objects or object classes can be recognized, usually together with their 2D positions in the image or 3D poses in the scene. Blippar, Google Goggles and LikeThat provide stand-alone programs that illustrate this functionality.
• Identification – an individual instance of an object is recognized. Examples include identification of a specific person's face or fingerprint, identification of handwritten digits, or identification of a specific vehicle.
• Detection – the image data are scanned for a specific condition. Examples include detection of possible abnormal cells or tissues in medical images or detection of a vehicle in an automatic road toll system. Detection based on relatively simple and fast computations is sometimes used for finding smaller regions of interesting image data which can be further analyzed by more computationally demanding techniques to produce a correct interpretation.

Currently, the best algorithms for such tasks are based on convolutional neural networks. An illustration of their capabilities is given by the ImageNet Large Scale Visual Recognition Challenge; this is a benchmark in object classification and detection, with millions of images and 1000 object classes used in the competition.^[28] Performance of convolutional neural networks on the ImageNet tests is now close to that of humans.^[28] The best algorithms still struggle with objects that are small or thin, such as a small ant on a stem of a flower or a person holding a quill in their hand. They also have trouble with images that have been distorted with filters (an increasingly common phenomenon with modern digital cameras). By contrast, those kinds of images rarely trouble humans. Humans, however, tend to have trouble with other issues. For example, they are not good at classifying objects into fine-grained classes, such as the particular breed of dog or species of bird, whereas convolutional neural networks handle this with ease^{[citation needed]}.

Several specialized tasks based on recognition exist, such as:

• Content-based image retrieval – finding all images in a larger set of images which have a specific content. The content can be specified in different ways, for example in terms of similarity relative a target image (give me all images similar to image X), or in terms of high-level search criteria given as text input (give me all images which contain many houses, are taken during winter, and have no cars in them).

Computer vision for people counter purposes in public places, malls, shopping centres

• Pose estimation – estimating the position or orientation of a specific object relative to the camera. An example application for this technique would be assisting a robot arm in retrieving objects from a conveyor belt in an assembly line situation or picking parts from a bin.
• Optical character recognition (OCR) – identifying characters in images of printed or handwritten text, usually with a view to encoding the text in a format more amenable to editing or indexing (e.g. ASCII).
• 2D code reading – reading of 2D codes such as data matrix and QR codes.
• Facial recognition
• Shape Recognition Technology (SRT) in people counter systems differentiating human beings (head and shoulder patterns) from objects

Motion analysis

Several tasks relate to motion estimation where an image sequence is processed to produce an estimate of the velocity either at each points in the image or in the 3D scene, or even of the camera that produces the images. Examples of such tasks are:

• Egomotion – determining the 3D rigid motion (rotation and translation) of the camera from an image sequence produced by the camera.
• Tracking – following the movements of a (usually) smaller set of interest points or objects (e.g., vehicles, humans or other organisms^[27]) in the image sequence.
• Optical flow – to determine, for each point in the image, how that point is moving relative to the image plane, i.e., its apparent motion. This motion is a result both of how the corresponding 3D point is moving in the scene and how the camera is moving relative to the scene.

Scene reconstruction

Given one or (typically) more images of a scene, or a video, scene reconstruction aims at computing a 3D model of the scene. In the simplest case the model can be a set of 3D points. More sophisticated methods produce a complete 3D surface model. The advent of 3D imaging not requiring motion or scanning, and related processing algorithms is enabling rapid advances in this field. Grid-based 3D sensing can be used to acquire 3D images from multiple angles. Algorithms are now available to stitch multiple 3D images together into point clouds and 3D models.^[19]

Image restoration

The aim of image restoration is the removal of noise (sensor noise, motion blur, etc.) from images. The simplest possible approach for noise removal is various types of filters such as low-pass filters or median filters. More sophisticated methods assume a model of how the local image structures look, to distinguish them from noise. By first analysing the image data in terms of the local image structures, such as lines or edges, and then controlling the filtering based on local information from the analysis step, a better level of noise removal is usually obtained compared to the simpler approaches.

An example in this field is inpainting.

System methods

The organization of a computer vision system is highly application-dependent. Some systems are stand-alone applications that solve a specific measurement or detection problem, while others constitute a sub-system of a larger design which, for example, also contains sub-systems for control of mechanical actuators, planning, information databases, man-machine interfaces, etc. The specific implementation of a computer vision system also depends on whether its functionality is pre-specified or if some part of it can be learned or modified during operation. Many functions are unique to the application. There are, however, typical functions that are found in many computer vision systems.

• Image acquisition – A digital image is produced by one or several image sensors, which, besides various types of light-sensitive cameras, include range sensors, tomography devices, radar, ultra-sonic cameras, etc. Depending on the type of sensor, the resulting image data is an ordinary 2D image, a 3D volume, or an image sequence. The pixel values typically correspond to light intensity in one or several spectral bands (gray images or colour images), but can also be related to various physical measures, such as depth, absorption or reflectance of sonic or electromagnetic waves, or nuclear magnetic resonance.^[24]
• Pre-processing – Before a computer vision method can be applied to image data in order to extract some specific piece of information, it is usually necessary to process the data in order to assure that it satisfies certain assumptions implied by the method. Examples are:
  • Re-sampling to assure that the image coordinate system is correct.
  • Noise reduction to assure that sensor noise does not introduce false information.
  • Contrast enhancement to assure that relevant information can be detected.
  • Scale space representation to enhance image structures at locally appropriate scales.
• Feature extraction – Image features at various levels of complexity are extracted from the image data.^[24] Typical examples of such features are:
  • Lines, edges and ridges.
  • Localized interest points such as corners, blobs or points.

More complex features may be related to texture, shape or motion.

• Detection/segmentation – At some point in the processing a decision is made about which image points or regions of the image are relevant for further processing.^[24] Examples are:
  • Selection of a specific set of interest points.
  • Segmentation of one or multiple image regions that contain a specific object of interest.
  • Segmentation of image into nested scene architecture comprising foreground, object groups, single objects or salient object^[29] parts (also referred to as spatial-taxon scene hierarchy),^[30] while the visual salience is often implemented as spatial and temporal attention.
  • Segmentation or co-segmentation of one or multiple videos into a series of per-frame foreground masks, while maintaining its temporal semantic continuity.^[31][32]
• High-level processing – At this step the input is typically a small set of data, for example a set of points or an image region which is assumed to contain a specific object.^[24] The remaining processing deals with, for example:
  • Verification that the data satisfy model-based and application-specific assumptions.
  • Estimation of application-specific parameters, such as object pose or object size.
  • Image recognition – classifying a detected object into different categories.
  • Image registration – comparing and combining two different views of the same object.
• Decision making Making the final decision required for the application,^[24] for example:
  • Pass/fail on automatic inspection applications.
  • Match/no-match in recognition applications.
  • Flag for further human review in medical, military, security and recognition applications.

Image-understanding systems

Image-understanding systems (IUS) include three levels of abstraction as follows: low level includes image primitives such as edges, texture elements, or regions; intermediate level includes boundaries, surfaces and volumes; and high level includes objects, scenes, or events. Many of these requirements are entirely topics for further research.

The representational requirements in the designing of IUS for these levels are: representation of prototypical concepts, concept organization, spatial knowledge, temporal knowledge, scaling, and description by comparison and differentiation.

While inference refers to the process of deriving new, not explicitly represented facts from currently known facts, control refers to the process that selects which of the many inference, search, and matching techniques should be applied at a particular stage of processing. Inference and control requirements for IUS are: search and hypothesis activation, matching and hypothesis testing, generation and use of expectations, change and focus of attention, certainty and strength of belief, inference and goal satisfaction.^[33]

Hardware

New iPad includes lidar sensor

There are many kinds of computer vision systems; however, all of them contain these basic elements: a power source, at least one image acquisition device (camera, ccd, etc.), a processor, and control and communication cables or some kind of wireless interconnection mechanism. In addition, a practical vision system contains software, as well as a display in order to monitor the system. Vision systems for inner spaces, as most industrial ones, contain an illumination system and may be placed in a controlled environment. Furthermore, a completed system includes many accessories such as camera supports, cables and connectors.

Most computer vision systems use visible-light cameras passively viewing a scene at frame rates of at most 60 frames per second (usually far slower).

A few computer vision systems use image-acquisition hardware with active illumination or something other than visible light or both, such as structured-light 3D scanners, thermographic cameras, hyperspectral imagers, radar imaging, lidar scanners, magnetic resonance images, side-scan sonar, synthetic aperture sonar, etc. Such hardware captures "images" that are then processed often using the same computer vision algorithms used to process visible-light images.

While traditional broadcast and consumer video systems operate at a rate of 30 frames per second, advances in digital signal processing and consumer graphics hardware has made high-speed image acquisition, processing, and display possible for real-time systems on the order of hundreds to thousands of frames per second. For applications in robotics, fast, real-time video systems are critically important and often can simplify the processing needed for certain algorithms. When combined with a high-speed projector, fast image acquisition allows 3D measurement and feature tracking to be realised.^[34]

Egocentric vision systems are composed of a wearable camera that automatically take pictures from a first-person perspective.

As of 2016, vision processing units are emerging as a new class of processor, to complement CPUs and graphics processing units (GPUs) in this role.^[35]

See also

• Computational imaging
• Computational photography
• Machine vision glossary
• Space mapping
• Teknomo–Fernandez algorithm
• Visual system
• Visual perception
• Vision science
• Egocentric vision
• Visual agnosia
• Computer audition

Lists

• List of computer vision topics
• List of emerging technologies
• Outline of artificial intelligence
• Outline of computer vision

References

1. Dana H. Ballard; Christopher M. Brown (1982). Computer Vision. Prentice Hall. ISBN 978-0-13-165316-0.
2. Huang, T. (1996-11-19). Vandoni, Carlo, E (ed.). Computer Vision : Evolution And Promise (PDF). 19th CERN School of Computing. Geneva: CERN. pp. 21–25. doi:10.5170/CERN-1996-008.21. ISBN 978-9290830955.
3. Milan Sonka; Vaclav Hlavac; Roger Boyle (2008). Image Processing, Analysis, and Machine Vision. Thomson. ISBN 978-0-495-08252-1.
4. Reinhard Klette (2014). Concise Computer Vision. Springer. ISBN 978-1-4471-6320-6.
5. Linda G. Shapiro; George C. Stockman (2001). Computer Vision. Prentice Hall. ISBN 978-0-13-030796-5.
6. Tim Morris (2004). Computer Vision and Image Processing. Palgrave Macmillan. ISBN 978-0-333-99451-1.
7. Bernd Jähne; Horst Haußecker (2000). Computer Vision and Applications, A Guide for Students and Practitioners. Academic Press. ISBN 978-0-13-085198-7.
8. David A. Forsyth; Jean Ponce (2003). Computer Vision, A Modern Approach. Prentice Hall. ISBN 978-0-13-085198-7.
9. http://www.bmva.org/visionoverview Archived 2017-02-16 at the Wayback Machine The British Machine Vision Association and Society for Pattern Recognition Retrieved February 20, 2017
10. Murphy, Mike. "Star Trek's "tricorder" medical scanner just got closer to becoming a reality".
11. Richard Szeliski (30 September 2010). Computer Vision: Algorithms and Applications. Springer Science & Business Media. pp. 10–16. ISBN 978-1-84882-935-0.
12. Papert, Seymour (1966-07-01). "The Summer Vision Project". MIT AI Memos (1959 - 2004). hdl:1721.1/6125.
13. Margaret Ann Boden (2006). Mind as Machine: A History of Cognitive Science. Clarendon Press. p. 781. ISBN 978-0-19-954316-8.
14. Takeo Kanade (6 December 2012). Three-Dimensional Machine Vision. Springer Science & Business Media. ISBN 978-1-4613-1981-8.
15. Nicu Sebe; Ira Cohen; Ashutosh Garg; Thomas S. Huang (3 June 2005). Machine Learning in Computer Vision. Springer Science & Business Media. ISBN 978-1-4020-3274-5.
16. William Freeman; Pietro Perona; Bernhard Scholkopf (2008). "Guest Editorial: Machine Learning for Computer Vision". International Journal of Computer Vision. 77 (1): 1. doi:10.1007/s11263-008-0127-7. ISSN 1573-1405.
17. Steger, Carsten; Markus Ulrich; Christian Wiedemann (2018). Machine Vision Algorithms and Applications (2nd ed.). Weinheim: Wiley-VCH. p. 1. ISBN 978-3-527-41365-2. Retrieved 2018-01-30.
18. Murray, Don, and Cullen Jennings. "Stereo vision based mapping and navigation for mobile robots." Proceedings of International Conference on Robotics and Automation. Vol. 2. IEEE, 1997.
19. Soltani, A. A.; Huang, H.; Wu, J.; Kulkarni, T. D.; Tenenbaum, J. B. (2017). "Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes With Deep Generative Networks". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: 1511–1519. doi:10.1109/CVPR.2017.269. hdl:1721.1/126644.
20. Turek, Fred (June 2011). "Machine Vision Fundamentals, How to Make Robots See". NASA Tech Briefs Magazine. 35 (6). pages 60–62
21. "The Future of Automated Random Bin Picking".
22. Chervyakov, N. I.; Lyakhov, P. A.; Deryabin, M. A.; Nagornov, N. N.; Valueva, M. V.; Valuev, G. V. (2020). "Residue Number System-Based Solution for Reducing the Hardware Cost of a Convolutional Neural Network". Neurocomputing: 439–453. doi:10.1016/j.neucom.2020.04.018. Convolutional neural networks (CNNs) represent deep learning architectures that are currently used in a wide range of applications, including computer vision, speech recognition, identification of albuminous sequences in bioinformatics, production control, time series analysis in finance, and many others.
23. Wäldchen, Jana; Mäder, Patrick (2017-01-07). "Plant Species Identification Using Computer Vision Techniques: A Systematic Literature Review". Archives of Computational Methods in Engineering. 25 (2): 507–543. doi:10.1007/s11831-016-9206-z. ISSN 1134-3060. PMC 6003396. PMID 29962832.
24. E. Roy Davies (2005). Machine Vision: Theory, Algorithms, Practicalities. Morgan Kaufmann. ISBN 978-0-12-206093-9.
25. Ando, Mitsuhito; Takei, Toshinobu; Mochiyama, Hiromi (2020-03-03). "Rubber artificial skin layer with flexible structure for shape estimation of micro-undulation surfaces". ROBOMECH Journal. 7 (1): 11. doi:10.1186/s40648-020-00159-0. ISSN 2197-4225.
26. Choi, Seung-hyun; Tahara, Kenji (2020-03-12). "Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors". ROBOMECH Journal. 7 (1): 14. doi:10.1186/s40648-020-00162-5. ISSN 2197-4225.
27. Bruijning, Marjolein; Visser, Marco D.; Hallmann, Caspar A.; Jongejans, Eelke; Golding, Nick (2018). "trackdem: Automated particle tracking to obtain population counts and size distributions from videos in r". Methods in Ecology and Evolution. 9 (4): 965–973. doi:10.1111/2041-210X.12975. ISSN 2041-210X.
28. Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; Berg, Alexander C. (December 2015). "ImageNet Large Scale Visual Recognition Challenge". International Journal of Computer Vision. 115 (3): 211–252. doi:10.1007/s11263-015-0816-y. hdl:1721.1/104944. ISSN 0920-5691.
29. A. Maity (2015). "Improvised Salient Object Detection and Manipulation". arXiv:1511.02999 [cs.CV].
30. Barghout, Lauren. "Visual Taxometric Approach to Image Segmentation Using Fuzzy-Spatial Taxon Cut Yields Contextually Relevant Regions." Information Processing and Management of Uncertainty in Knowledge-Based Systems. Springer International Publishing, 2014.
31. Liu, Ziyi; Wang, Le; Hua, Gang; Zhang, Qilin; Niu, Zhenxing; Wu, Ying; Zheng, Nanning (2018). "Joint Video Object Discovery and Segmentation by Coupled Dynamic Markov Networks" (PDF). IEEE Transactions on Image Processing. 27 (12): 5840–5853. Bibcode:2018ITIP...27.5840L. doi:10.1109/tip.2018.2859622. ISSN 1057-7149. PMID 30059300. S2CID 51867241. Archived from the original (PDF) on 2018-09-07. Retrieved 2018-09-14.
32. Wang, Le; Duan, Xuhuan; Zhang, Qilin; Niu, Zhenxing; Hua, Gang; Zheng, Nanning (2018-05-22). "Segment-Tube: Spatio-Temporal Action Localization in Untrimmed Videos with Per-Frame Segmentation" (PDF). Sensors. 18 (5): 1657. doi:10.3390/s18051657. ISSN 1424-8220. PMC 5982167. PMID 29789447.
33. Shapiro, Stuart C. (1992). Encyclopedia of Artificial Intelligence, Volume 1. New York: John Wiley & Sons, Inc. pp. 643–646. ISBN 978-0-471-50306-4.
34. Kagami, Shingo (2010). "High-speed vision systems and projectors for real-time perception of the world". 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. 2010. pp. 100–107. doi:10.1109/CVPRW.2010.5543776. ISBN 978-1-4244-7029-7. S2CID 14111100.
35. Seth Colaner (January 3, 2016). "A Third Type Of Processor For VR/AR: Movidius' Myriad 2 VPU". www.tomshardware.com.

Further reading

• David Marr (1982). Vision. W. H. Freeman and Company. ISBN 978-0-7167-1284-8.
• Azriel Rosenfeld; Avinash Kak (1982). Digital Picture Processing. Academic Press. ISBN 978-0-12-597301-4.
• Barghout, Lauren; Lawrence W. Lee (2003). Perceptual information processing system. U.S. Patent Application 10/618,543. ISBN 978-0-262-08159-7.
• Berthold K.P. Horn (1986). Robot Vision. MIT Press. ISBN 978-0-262-08159-7.
• Michael C. Fairhurst (1988). Computer Vision for robotic systems. Prentice Hall. ISBN 978-0-13-166919-2.
• Olivier Faugeras (1993). Three-Dimensional Computer Vision, A Geometric Viewpoint. MIT Press. ISBN 978-0-262-06158-2.
• Tony Lindeberg (1994). Scale-Space Theory in Computer Vision. Springer. ISBN 978-0-7923-9418-1.
• James L. Crowley and Henrik I. Christensen (Eds.) (1995). Vision as Process. Springer-Verlag. ISBN 978-3-540-58143-7.
• Gösta H. Granlund; Hans Knutsson (1995). Signal Processing for Computer Vision. Kluwer Academic Publisher. ISBN 978-0-7923-9530-0.
• Reinhard Klette; Karsten Schluens; Andreas Koschan (1998). Computer Vision – Three-Dimensional Data from Images. Springer, Singapore. ISBN 978-981-3083-71-4.
• Emanuele Trucco; Alessandro Verri (1998). Introductory Techniques for 3-D Computer Vision. Prentice Hall. ISBN 978-0-13-261108-4.
• Bernd Jähne (2002). Digital Image Processing. Springer. ISBN 978-3-540-67754-3.
• Richard Hartley and Andrew Zisserman (2003). Multiple View Geometry in Computer Vision. Cambridge University Press. ISBN 978-0-521-54051-3.
• Gérard Medioni; Sing Bing Kang (2004). Emerging Topics in Computer Vision. Prentice Hall. ISBN 978-0-13-101366-7.
• R. Fisher; K Dawson-Howe; A. Fitzgibbon; C. Robertson; E. Trucco (2005). Dictionary of Computer Vision and Image Processing. John Wiley. ISBN 978-0-470-01526-1.
• Nikos Paragios and Yunmei Chen and Olivier Faugeras (2005). Handbook of Mathematical Models in Computer Vision. Springer. ISBN 978-0-387-26371-7.
• Wilhelm Burger; Mark J. Burge (2007). Digital Image Processing: An Algorithmic Approach Using Java. Springer. ISBN 978-1-84628-379-6.
• Pedram Azad; Tilo Gockel; Rüdiger Dillmann (2008). Computer Vision – Principles and Practice. Elektor International Media BV. ISBN 978-0-905705-71-2.
• Richard Szeliski (2010). Computer Vision: Algorithms and Applications. Springer-Verlag. ISBN 978-1848829343.
• J. R. Parker (2011). Algorithms for Image Processing and Computer Vision (2nd ed.). Wiley. ISBN 978-0470643853.
• Richard J. Radke (2013). Computer Vision for Visual Effects. Cambridge University Press. ISBN 978-0-521-76687-6.
• Nixon, Mark; Aguado, Alberto (2019). Feature Extraction and Image Processing for Computer Vision (4th ed.). Academic Press. ISBN 978-0128149768.

External links

• USC Iris computer vision conference list
• Computer vision papers on the web A complete list of papers of the most relevant computer vision conferences.
• Computer Vision Online News, source code, datasets and job offers related to computer vision.
• Keith Price's Annotated Computer Vision Bibliography
• CVonline Bob Fisher's Compendium of Computer Vision.
• British Machine Vision Association Supporting computer vision research within the UK via the BMVC and MIUA conferences, Annals of the BMVA (open-source journal), BMVA Summer School and one-day meetings
• Computer Vision Container, Joe Hoeller GitHub: Widely adopted open-source container for GPU accelerated computer vision applications. Used by researchers, universities, private companies as well as the U.S. Gov't.