IvoVondrak

Neurex Expert System

The effective decision support should be based on both getting information from databases and knowledge bases. If the first one is quite successfully solved by information systems the usage of knowledge stored in knowledge bases and obtained via inference engines of expert systems is not on the level originally expected. Neural Networks and the age of connectionism bring a new view on this problem, it means, how to obtain reliable knowledge necessary to decision support and how it can be used later in user-friendly way. Usual expert systems with rule-based and/or frame-based knowledge are linked with problems of a creation of the reliable knowledge base. Neural networks can solve these difficulties. It is possible to create knowledge base both without experts (only by the use of data collections describing the solved area) and with experts whose knowledge can be checked during the process of learning.The effort to build such reliable expert system led to the creation of the expert system Neurex based on the technology of multi-layered neural network. The structure (topology) of such network is determined by the layers of neurons where neurons in adjacent layers are connected each other. The connections between neurons are valued by the weights – w and functionality of each neuron is given by its activation dynamics represented by transfer function S. Output signal of the neuron is given in such a way according formula

y = S(w.x)

The scalar product between parentheses represents input of the neuron given by the sum of weighted output signals x from all neurons in the lower layer. In such model of the neural network the signal spreads in the direction from the lower layer to the upper one.The creation of the knowledge within neural network employs the design principles as follows:

• Input and output facts are represented by the neurons of the input and output layers whose states of excitation express truth values (excitation 1 means True and 0 means False) or values from the given range (maximum means excitation 1 and minimum excitation 0).
• Knowledge is encoded into weights (strengths) of interconnections between neurons (Circuit-based Knowledge). Definition of these values is solved via process of learning on the training set

T= { [Ai, Bi, Ci, Di, Ei, Gi | Xi,Yi] } for i = 1 … n,

where n is a number of patterns describing solved area. The patterns can be obtained by measurement as well as by experts. The usage of the learnt (adapted) neural network is given by the excitation of the neurons from the input layer and then the signal is transferred through the other layer as far as the output layer. Obtained excitations of the output neurons are related to the values of the output facts. The first and main advantage of such circuit-based knowledge base is that it is created by the process of learning. Secondly, the neural network can generalize the knowledge, it means that it can find solution for input that is not included in the training set. The last advantage is the high speed of inference. All above mentioned principles are used in this expert system Neurex that is based on the multi-layered neural network where activation dynamics of the neurons are solved via sigmoidal function. The slope of this sigmoid and the threshold of each neuron are considered as the parameters of the network what means that they are adapted, too. The result of the learning process is so called heterogeneous neural network where neurons differ by the shape of their activation dynamics and by their thresholds.

Example of training set for a simple medical diagnosis (5 inputs and 3 outputs)

Fever	Cough	Headache	Tiredness	Night Sweat	Pneumonia	Flu	Cold
0	1	1	0	0	0	0	1
1	1	1	1	0	0	1	0
1	1	0	1	1	1	0	0

The process of the expert system’s usage can be described in the following way:

1. Definition of the topology of the neural network. In this section there is a number of input and output facts defined as well as number of hidden layers is determined determined.
2. Then input and output facts (attributes) are formulated. It means that each fact is connected to one neuron in input resp. output layer. The range of values for each of attributes is defined.
3. Definition of the training set. Patterns are entered by truth values (e.g. 0-1 or 0-100%) or by values from the range given in previous steps.
4. Phase of learning of the network. The weights of the connections (synapses) between neurons, slopes of sigmoids and thresholds of the neurons are computed by “Back propagation”(BP) method (Options enables to define parameters for that process with are learning rate and number of learning cycles) based on patterns from training set. These values represent a memory (base of knowledge) of the expert system. The result pf learning process is displayed using mean squared error, and the index of the worst patterned and its error in [%] si displayed.
5. Consultation with the system. This phase is based on the definition of the values of the input facts after which the values of output facts are deduced immediately.

I have also implemented backpropagation multilayered neural net in Python. You can play with it using Jupyter notebook here.

Zpět na hlavní stránku