Transcript Particle Swarm Optimization mini tutorial

Particle Swarm optimisation: A mini
tutorial
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
The “inventors” (1)
Russell
Eberhart
eberhart@engr.iupui.edu
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
The “inventors” (2)
James
Kennedy
Kennedy_Jim@bls.gov
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Part 1: United we stand
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Cooperation example
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Initialization. Positions and velocities
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Neighbourhoods
geographical
2002-04-24
social
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
The circular neighbourhood
1
Particle 1’s 3neighbourhood
2
8
3
7
Virtual circle
4
6
5
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Psychosocial compromise
My best
perf.
pi
Here I am!
x
pg
The best perf. of
my neighbours
v
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
The historical algorithm
At each time step t
for each particle
for each component d
vd t  1 
update v t 
 d
the 
 rand0, 1  pi , d  xd t 

velocity
 rand0,  2  p g , d  xd t 

then move x(t  1)  xt   vt  1
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Random proximity
Hyperparallelepiped => Biased
pi
x
pg
v
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Animated illustration
Global
optimum
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Part 2: How to choose parameters
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Type 1” form
v(t  1)   (v(t )   ( p  x(t )))

 x(t  1)  v(t  1)  x(t )
with
  rand(0, 1 )  rand(0,  2 )   '1  '2
p
 '1 pi   ' 2 p g
 '1  ' 2
2

for   4


    2   2  4


else 
2002-04-24
Particle Swarm optimisation
Usual values:
=1
=4.1
=> =0.73
swarm size=20
hood size=3
Maurice.Clerc@WriteMe.com
5D complex space

A 3D section
} Convergence
6
5
4
}
3
-2
2
-1
-4
1
-2
0
00
2
Re(v)
2002-04-24
4
Particle Swarm optimisation
Non
divergence
1
2
Re(y)
Maurice.Clerc@WriteMe.com
Move in a 2D section (attractor)
Im(v)
0.8
0.6
0.4
0.2
-1
-0.5
00
0.5
-0.2
-0.4
1
Re(v)
-0.6
-0.8
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Some functions ...
Griewank
Rastrigin
Rosenbrock
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
... and some results
Optimum=0, dimension=30
Best result after 40 000 evaluations
30D function
PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [±300]
0.003944
0.4033
Rastrigin [±5]
82.95618
46.4689
Rosenbrock [±10] 50.193877
2002-04-24
Particle Swarm optimisation
1610.359
Maurice.Clerc@WriteMe.com
Beat the swarm!
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Part 3: Beyond real numbers
1
8
2002-04-24
0
2
0
1
3
1
2
4
0
2
3
0
5
1
3
4
1
6
4
0
2
2
1
3
3
2
4
Particle Swarm optimisation
Bingo!
8
Maurice.Clerc@WriteMe.com
Minimun requirements
Comparing positions in the
search space H
( x, x' )  H  H ,  f x   f x'   f x   f x'
Algebraic operators

coefficient , velocity 
velocity

velocity, velocity 
velocity

 position, position 
velocity

 position, velocity 
position
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Pseudo code form
velocity = pos_minus_pos(position1, position2)
}
velocity = linear_combin(,velocity1,,velocity2)
position = pos_plus_vel(position, velocity)
algebraic
operators
(position,velocity) = confinement(positiont+1,positiont)
=>
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Fifty-fifty


xi  1...N

i  j  xi  x j
D / 2
D

xi
 xi 

 1
D / 21
 

N=100, D=20. Search space: [1,N]D
105 evaluations:
63+90+16+54+71+20+23+60+38+15
=
12+48+13+51+36+42+86+26+57+79 (=450)
2002-04-24
Particle Swarm optimisation

Maurice.Clerc@WriteMe.com
Knapsack


x  1...N
 i

i  j  xi  x j



xi  S
iI , I  D, I 1, N 

 

N=100, D=10, S=100,
870 evaluations:
run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12, 17)
run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Graph Coloring Problem
1
2
0
5
5
3
5
1
=
2002-04-24
+
2
-1
1
-1
0
-3
2
1
1
4
1
5
4
5
Particle Swarm optimisation
2
5
Maurice.Clerc@WriteMe.com
The Tireless Traveller
Example of position: X=(5,3,4,1,2,6)
Example of velocity: v=((5,3),(2,5),(3,1))
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Apple trees
Best position
Swarm size=3
n1
n3
Evaluation n1
0
3
6
1
3
2
3
7
n2
0
4
11
n3
17
10
6
7 6
n2
f  n1  n2 2  n2  n3 2
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Part 4: Some variants
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Unbiased random proximity
Hyperparallelepiped => Biased
Hypersphere vs hypercube
1.2
pi
x
1
pg
Volume
0.8
0.6
0.4
0.2
v
0
1
3
5
7
9
11 13 15 17 19 21 23 25 27 29
Dimension
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Clusters and queens
Each particle is weighted by its
perf.
Dynamic clustering
2002-04-24
Particle Swarm optimisation
Centroids = queens =
temporary new
“particles”
Maurice.Clerc@WriteMe.com
Think locally, act locally (Adaptive
versions)
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Adaptive swarm size
There has been enough
improvement
although I'm the worst
I'm the best
but there has been not enough
improvement
2002-04-24
Particle Swarm optimisation
I try to kill myself
I try to generate a
new particle
Maurice.Clerc@WriteMe.com
Adaptive coefficients
v
rand(0…b)(p-x)
The better I
am, the more I
follow my own
way
2002-04-24
Particle Swarm optimisation
The better is my best
neighbour, the more
I tend to go towards
him
Maurice.Clerc@WriteMe.com
Energies: classical process
Rosenbrock 2D. Swarm size=20, constant coefficients
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Energies: adaptive process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Part 5: Real applications (hybrid)
Medical diagnosis
Electrical generator
2002-04-24
Industrial mixer
Electrical vehicle
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Real applications (stand alone)
Cockshott A. R., Hartman B. E., "Improving the fermentation medium for
Echinocandin B production. Part II: Particle swarm optimization", Process
biochemistry, vol. 36, 2001, p. 661-669.
He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting
Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE
International Conference on Evolutionary Computation, Anchorage, Alaska,
USA, 1998.
Secrest B. R., Traveling Salesman Problem for Surveillance Mission using
Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of
Technology, 2001.
Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for
Reactive Power and Voltage Control considering Voltage Security
Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
To know more
THE site:
Particle Swarm Central, http://www.particleswarm.net
Self advert
Clerc M., Kennedy J., "The Particle Swarm-Explosion,
Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary
Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire.
Principes et pratique", Hermès, Techniques et
Science de l'Informatique, 2002.
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Appendix
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Canonical form
vt  1  vt     p  xt 

 xt  1  xt   vt  1
yt   p  xt 
M
   vt 
 vt  1  1
 y t  1   1 1     y t 

 


Eigen values e1 and e2
v t 1  v t   
 yt

  y t 
yt  1   vt   
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Constriction
Constriction coefficients
2
2















2





2





 1 
2

2




 4



       2  2     2       2
 2 

2     2  4
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
Convergence criterion


  1e1  1
 1  1

 

  2 e 2  1 
  2 e2  1
  2 e 2
3.5
3
2.5
2
1.5
1
0.5
1
2002-04-24
2
3

Particle Swarm optimisation
4
5
6
Maurice.Clerc@WriteMe.com
Magic Square (1)
 m1,1

 
m
 D ,1

D 

m D, D 


mi , j
m1,


mi , j  x j  i 1

mi , j  1 N 

mi , j  mk ,l
2002-04-24
D 1 
  m
i 1

D



D
i, j
j 1
D 1 
 
j 1
D
 mi 1, j






mi , j  mi , j 1

 i 1
2





0
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
2
Magic Square (2)
55 30 68
42 49 62
56 74 23
30 61 53
89 32 23
25 51 68
43 51 78
75 33 64
54 88 30
80 3 30
22 72 19
11 38 64
D=3x3, N=100
10 runs
13430 evaluations
27 96 39
73 40 49
62 26 74
2002-04-24
50 43 67
58 55 47
52 62 4
10 solutions
22 70 58
40 75 35
88 5 57
Particle Swarm optimisation
18 25 59
32 53 17
52 24 26
65 28 64
63 55 39
29 74 54
50 65 68
69 42 72
64 76 43
Maurice.Clerc@WriteMe.com
Non linear system
2
2

 x1  x2  1  0


sin10x1   x2  0
Search space
[0,1]2
2002-04-24
1 run
143 evaluations
1 solution
10 runs
1430 evaluations
3 solutions
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com
2002-04-24
Particle Swarm optimisation
Maurice.Clerc@WriteMe.com