Increasing the power of a von Neumann computer is limited by the limits of Moore's law. As it has become increasingly more difficult to crowd more and more electronic components onto a single microprocessor, new types of computers have been invented to generate greater and greater computing power. We shall consider:
A parallel computer may have more than one CPU controlled by a single operation system. Some problems are intrinsically sequential and there is no possible gain from having more than one CPU. Unlike a Von Neumann design where all designs are variations on the basic principles, a parallel computer has many alternative designs, each of which is best suited for a particular class of problems. Alternatives cover:
In the past parallel computers tend to be supercomputers at the limit of computational capabilities. An example is IBM's Deep Blue computer that beat the world champion in chess, Garry Kasparov in 1997. Again, this is more a matter of brute force in looking at very large numbers of alternatives than 'thinking" like a chess player. However, now consumers are able to purchase dual core computers from any computer manufacturer. The dual core processors provide an increase in overall performance, but have the greatest value under multi-tasking conditions.
A multi-core system is where several processors are placed on the central processing unit instead of having multiple CPUs. Currently the two main producers of multi-core systems for the consumer market are AMD and Intel. One concern to the manufacturers of multi-core systems is software licensing. Many software applications are not licensed by the computer but instead are licensed by the processor. For now, Microsoft has agreed to treat multi-core CPUs as single processor rather than as multiple processors. Multi-core parallel computing also has the advantage of being easier to integrate into the current PC manufacturing complex. This is because multi-core CPUs fit into the current sockets of motherboards, which means that there does not have to be a reorganization of the production method to provide parallel computing to the masses.
One early development in parallel processors was the development of supercomputers by Seymour Cray and Control Data that had a very small number of very powerful processors that had hardware for array operations, such as a vector dot product. Cray was bought out by SGI and then came back as a separate firm, Cray. These types of parallel processors are primarily used in scientific computing. Remember there is no limit to the power of a computer that weather forecasters can use. The more powerful the computer the finer the grid of the biosphere they will use in the computations. Parallel computers are also used as servers in situations where the server has to process a very large number of requests.
An alternative approach for business computing such as internet servers is using large numbers of PC like processors. Here there is an economic concept called scalable computing. Suppose a firm buys a computer and as demand on the computer increases the firm needs a bigger computer. The old approach would be to replace the current computer with a bigger more powerful computer. With parallel processing based on PC type processors the new approach is to expand the computer with the addition of additional clusters of PC type processors. Among the several firms taking this approach are HP and Sun. Server computers are increasingly tending to be parallel processors. The technical problem to overcome in scalable computing is how to effectively cluster the PC type processors. The master at building supercomputers is IBM.
Parallel Computers
Distributed Processing
Distributed processing is processing that takes place in a network of computers controlled by separate operating systems. Coordinating distributed processing is more complicated than parallel processing because of the different operating systems.
Then came the explosion of personal computers. At first personal computers were standalone units on employees' desks. Because these PCs were not linked to the corporate mainframe, employees had a difficult time obtaining corporate data for their work. This difficulty created incentives to link these personal computers into networks.
A rapidly growing type of computer network is a client-server
network, which is a network of clients, which are PCs, MAC or
workstations used by employees, connected to a server which furnishes
clients with such things as huge disk drives, databases, or connections
to a network. Servers can be mainframes, minicomputers, workstations,
or powerful PCs. More than one server can supply services to the
clients. In creating a client-server network, it is frequently
efficient to replace an expensive corporate mainframe with a much less
costly, powerful workstation. This is known as downsizing. A
client-server network is a distributed processing system because some
processing takes place at each node. There are many physical
organizations and two control mechanisms.
Physical organization
Bus - single backbone, terminated at both ends
Ring - connects 1 host to the next, and the last to the first (picture
below is ring organizaiton)
Star - connects all devices to a central point(See below)
More advanced are the Extended Star,Hierarchical and Mesh -
interconnects all devices with all other devices
Control Mechanism
Token Passing - a single electronic “token” is passed around and
computers send data one at a time (only while possessing the token)Inefficient
Broadcast - each host sends data to all other hosts on the network on a
first come, first serve basis. More efficient (can not be used
on star organizaiton)
A simple type of client-server network where the clients are connected
in the digital analog of a party line former found in rural areas.
A client-server network has many advantages. If a client machine fails, the network remains operational. Such a network has tremendous computer power at a low cost because powerful workstations are as powerful as older mainframes at 1/10th cost. You can pick and choose hardware, software, and services from various vendors because these networks are open systems. Such systems can easily be expanded or modified to suit individual users and departments. The difficulties of such systems are that they are difficult to maintain, lack support tools, and require retraining the corporate programming staff.
Today the Internet is the largest distributed processing system in the world. A new development in distributed processing is grid computing where computations take place across a network where the individual computers are in different administrative domains. The economics of grid computing is to take advantage of idle time of computers in a network and sell the time to organizations who needs extra computer power for a fixed time span.
Cloud Computing: (added 7 Oct 08 by Andrew Ku
and Alfred Norman
An development beyond grid computing is Cloud Computing.
Definition: Cloud Computing is a style of computing in which the single unifying theme is an increasing reliance on the Internet to satisfy the computing needs-software, computational resources, and storage-- of users.
In general, it is a concept in which IT-related capabilities are provided “as a service” (software as a service, platform as a service, infrastructure as a service, etc…). The “cloud” is a metaphor for the Internet, which serves as the interconnect between the user and the technology infrastructure that supplies these services. Users access these services without any knowledge or control of the infrastructure behind them.
Cloud users store their data in the cloud, use programs located in the cloud, and execute the programs in the cloud. Firms offering cloud computing include Amazon, Google, IBM, Microsoft, Yahoo. A user performing cloud computing through, say Google, would use a subset of the computer and software resources owned or controlled by Google. Regardless of location, any connected computer user can access supercomputering power. Within a cloud, a business traveler traveling from London to New York could crunch financial forecasts, while another cloud user, a university student, writes up their thesis.
Economics:
For Users:
The main problem with cloud computing is infrastructure security,
specifically data loss or access loss.
For Providers:
Two problems with cloud computing:
Security: In order to have clients, sellers of cloud
computing services are going to have to make sure their clients
software operations in their cloud are secure.
Power Outage: A cloud provider must maintain power as
an outage could cripple a client.
Major Players:
Distributed Processing:
Artificial Neural Networks, ANN
An artificial neural network, ANN is a software simulation of a biological neural network, brain, run on a conventional computer. Because it would be much faster to have integrated circuits specifically designed as ANN, firms are creating special purpose, ANN ICs such as Accurate Automation Corporation's NNP line
Some definitions of ANN are:
A diagram of a simple feedforward ANN is shown below:
Inputs are entered at the input layer. Note each element in the input layer is connected to each element of the hidden layer. Depending in the weights of the inputs a particular element of the hidden layer activates if there sum is greater than the activation threshold for that element. Each element in the hidden layer is connected to each element in the output layer. Again depending on the weights of the output from the hidden layer the elements of the output layer fire if the sum is greater than the activation threshold.
ANN are not programmed like conventional computers rather they are trained either with or without a teacher. In supervised learning the ANN is subjected to a large number of training sets of data and the weights and thresholds are adjusted to minimize the error in its predications of the training set data. Any type of optimization algorithm can be used to train an ANN and researchers have explored the alternatives from simple gradient procedures to genetic algorithms. The model of the process is the final weights and thresholds of the various elements. It is not an explicit set of mathematical equations. In unsupervised learning the ANN simple processes the data sets discovering patterns. There are numerous technical problems to overcome in obtaining good performance from an ANN.
As NeuroDynamX points out their ANNs have numerous application in:
"Neural networks often outperform traditional approaches at pattern recognition, classification, data clustering and modeling of nonlinear relationships. DynaMind® Developer Pro lets you access neural network technology for a variety of applications.
While an ANN is a massively parallel processor, the term parallel processor usually refers to digital computers with more than one CPU controlled by one operating system and programmed with specific instructions. Hence, an ANN can be simulated in a parallel processor.
Neural Network
DNA Computer
An experimental computer type of computer that has the potential to
solve NP problems efficiently is a DNA computer. The first DNA computer
was created at University of Southern California by Len Adleman.
Earlier designs for DNA computers used ATP (a phosphate molecule that
provides energy for cell functions) to power the computer. A major
breakthrough occurred when Israeli scientists at the Weizmann Institute
used DNA as the computers energy source, software and hardware. This
allowed them to have 15,000 trillion DNA computers in the size of a
spoonful of liquid and it also allowed them to achieve over 1,000,000
times the energy efficiency of a normal PC. In terms of storage DNA
computers have enormous potential, since 1 cubic centimeter of DNA has
the potential to store over one trillion compact discs worth of data.
However currently the DNA computers are very elementary and are faced
with several limitations. These include only being able to answer
yes/no questions, and being unable to count. Their strengths are in
being able to solve many different task simultaneously rather then
solve one task very quickly, much like parallel computing On the
commercial front Olympus Optical has built the “first commercially
practical DNA computer that specialized in gene analysis.” They plan on
selling the abilities of this computer as a service to researchers. The
major developments of DNA computers are expected to be in computers
that will put in into one self, and be able to check for diseases and
cancers. In this regard, the Weizmann institute has developed a medical
kit capable of diagnosing cancers in a test tube, but they believe they
are still decades away from being able to deploy it in humans.
DNA Computer-References
Revised: 25 August 09