cameuh.net Archives

Installer un cluster GNBD/GFS/Oracle de quatre serveurs Redhat ES 3 + Oracle 9i

-> Brouillon

Introduction

Nous avons 4 serveurs, ora01, ora02, ora03 et ora04.

Chaque serveur va partager sa partition de 60GB via GNBD sur le réseau aux trois autres.

Chaque serveur va assembler les trois partitions réseaux + la sienne pour faire un volume GFS.

Chaque serveur fera tourner une instance d’Oracle cluster sur la partition GFS.

LVM

GFS a besoin d’une partition de 4MB pour stocker des informations de configuration. Nous allons donc installer LVM sur notre partition disque afin de pouvoir facilement découper de l’espace à cet effet.

vgscan
pvcreate /dev/cciss/c0d0p7
vgcreate vg /dev/cciss/c0d0p7
lvcreate -n meta -L4M vg
vgdisplay vg (pour savoir combien d'extents il reste, ici 1803)
lvcreate -n data -l1803 vg

GNBD

Sur chaque machine,

exporter les partitions

modprobe gnbd_serv
gnbd_export -d /dev/vg/meta -e ora01meta -c
gnbd_export -d /dev/vg/data -e ora01data -c

importer les partitions GNBD des serveurs

modprobe gnbd
gnbd_import -i ora01
gnbd_import -i ora02
gnbd_import -i ora04
gnbd_import -i ora03

Pool

apt-get install XFree86-xauth \
 XFree86-base-fonts \
 htmlview \
 PyXML \
 gnome-python2 \
 gnome-python2-canvas \
 pygtk2 \
 pygtk2-libglade

Installer les paquets GFS (les RPM sont également dispos chez CentOS http://ftp.freepark.org/pub/linux/distributions/centos/3/csgfs/i386/RPMS/)

rpm -hiv GFS-6.0.2.36-1.i686.rpm \
 GFS-modules-smp-6.0.2.36-1.i686.rpm \
 perl-Net-Telnet-3.03-2.noarch.rpm \
 rh-gfs-en-6.0-4.noarch.rpm \
 rh-cs-en-3-2.noarch.rpm \
 clumanager-1.2.34-3.i386.rpm \
 redhat-config-cluster-1.0.8-1.noarch.rpm
modprobe pool

Fichier de config

[root@ora01 root]# cat ora.cfg
poolname ora
subpools 1
subpool 0 128 4
pooldevice 0 0 /dev/gnbd/ora01
pooldevice 0 1 /dev/gnbd/ora02
pooldevice 0 2 /dev/gnbd/ora03
pooldevice 0 3 /dev/gnbd/ora04

Creation du pool

[root@ora01 root]# pool_tool -c ora.cfg
/dev/cciss/c0d0p7 has "EXT2/3 filesystem" on it.
Are you sure you want to overwrite "EXT2/3 filesystem" with a pool label on /dev/cciss/c0d0p7? (y/n)y
/dev/gnbd/ora02 has "EXT2/3 filesystem" on it.
Are you sure you want to overwrite "EXT2/3 filesystem" with a pool label on /dev/gnbd/ora02? (y/n)y
/dev/gnbd/ora03 has "EXT2/3 filesystem" on it.
Are you sure you want to overwrite "EXT2/3 filesystem" with a pool label on /dev/gnbd/ora03? (y/n)y
/dev/gnbd/ora04 has "EXT2/3 filesystem" on it.
Are you sure you want to overwrite "EXT2/3 filesystem" with a pool label on /dev/gnbd/ora04? (y/n)y
Pool label written successfully from pool.cfg
[root@ora01 root]#

Activation du pool

pool_assemble ora

Le CCA (Cluster Configuration Archive)

Il contient la configuration du cluster

mkdir ccs
cat > ccs/cluster.ccs %3c%3c EOF
cluster {
 name = "oracl"
 lock_gulm {
  servers = [ "ora01", "ora02", "ora03" ]
 }
}
EOF
cat > ccs/fence.ccs %3c%3c EOF
fence_devices {
 gnbd {
  agent = "fence_gnbd"
  server = "ora01"
  server = "ora02"
  server = "ora03"
  server = "ora04"
 }
}
EOF
cat > ccs/nodes.ccs %3c%3c EOF
nodes {
 ora01 {
  ip_interfaces {
   eth0 = "10.1.29.84"
  }
  fence {
   server {
    gnbd {
     ipaddr = "10.1.29.84"
    }
   }
  }
 }
 ora02 {
  ip_interfaces {
   eth0 = "10.1.29.85"
  }
  fence {
   server {
    gnbd {
     ipaddr = "10.1.29.85"
    }
   }
  }
 }
 ora03 {
  ip_interfaces {
   eth0 = "10.1.29.86"
  }
  fence {
   server {
    gnbd {
     ipaddr = "10.1.29.86"
    }
   }
  }
 }
 ora04 {
  ip_interfaces {
   eth0 = "10.1.29.87"
  }
  fence {
   server {
    gnbd {
     ipaddr = "10.1.29.87"
    }
   }
  }
 }
}
EOF

On crée ensuite le pool spécial CCA et on y place l’archive :

cat > cca.cfg %3c%3c EOF
poolname cca
subpools 1
subpool 0 128 4
pooldevice 0 0 /dev/gnbd/ora01meta
pooldevice 0 1 /dev/gnbd/ora02meta
pooldevice 0 2 /dev/gnbd/ora03meta
pooldevice 0 3 /dev/gnbd/ora04meta
EOF
pool_tool cca.cfg
pool_assemble
ccs_tool create ccs /dev/pool/cca

Serveur de lock

Lancer sur les trois machines :

lock_gulmd