Exporting ZFS filesystem to non-global zone

There are 2 ways zfs filesystem can be assigned to a non-global zone. 

Method 1:

1. Create a zfs filesystem on global zone 

     

        zfs create orapool/app

Where orapool - is the zpool name and app is the zfs filesystem name

2. Export the above the zfs filesystem to non-global zone called appzone

zonecfg -z appzone

[root@server1]# zonecfg -z appzone
zonecfg:appzone> add fs
zonecfg:appzone:fs> set type=zfs
zonecfg:appzone:fs> set special=orapool/app
zonecfg:appzone:fs> set dir=/zonefs
zonecfg:appzone:fs> end
zonecfg:appzone> commit
zonecfg:appzone> exit

In  the above method, non-global zone administrator does not have any control over the exported the filesystem, means can not set any attribute for the filesystem


Method 2:

In this method , the non-global zone administrator has complete control over the filesystem

[root@server1]# zonecfg -z appzone
zonecfg:appzone> add dataset
zonecfg:appzone:dataset> set name=orapool/app
zonecfg:appzone:dataset> end
zonecfg:appzone> commit
zonecfg:appzone> exit

Once its exported, you can login to non-global zone and go ahead with the creating any no of filesystem on the exported the dataset

For example on orapoo/app you need orapool/app/test , orapool/app/test1,  orapool/app/tes3

Use the following command to create the above filesystem on non-global zone

#zfs create  orapool/app/test

#zfs create orapool/app/test1

#zfs create orapool/app/test2

So second approach is always best when you need to create multiple mountpoints on a dataset on non-global zone.

Other info:

 

Realtime issue faced:

I came across this issue when i was trying to set the mountpoint properties of previously exported dataset from global zone. I was getting error message as "Dataset is busy , its already exported"

Reason:

 When you export a dataset from a global zone, it sets  dataset's zone attribute,  which does not get turned off even if you remove the dataset from zone configuration using zonefcg command. You can view all the attributes of zone using

#zfs get orapool/app

Solution:

Turn of the zone attribute

#zfs set zone=off orapool/app

 

 

Persitent Binding for HBA Card

Persistent binding : As the name implies persistent binding enables LUNs zoned from a Storage frame will be assigned with a specific target ID. Now next question which may pops up in your mind "Why do we need that"?

I don't have any strong answer for it. After googling, got few clarifications which are not much convincing.

Below are the few points which i feel could be a reasonable answer for the above question.

1. In a cluster environment where the same LUN is assigned to multiple machine should be configured with persistent target ID.

2. Its a recommended steps for emulex branded HBA card where drivers software are provided by Emulex and same applies for Q Logic branded card.

Here in this, i would be taking only Emulex and Qlogic card as these cards are the most commonly used card on Solaris Box.

NOTE : You do not need to configure persistent binding for Sun Branded Emulex and Qlogic card. 

So HBA cards are categorized as Sun-branded and Sun-non-branded.

Below are the tips to identify the sun-branded and non-branded qlogic and emulex card.

Emulex Card:

LPxxxxxx-S are SUN Emulex HBA Cards  (ie has a “-S” for Sun at end).

For Solaris 10 fcinfo hba-port would provide you the details and for previous version of solaris prtpicl command can be used to get the HBA details

If you see any of the following references they are NON-SUN HBA Cards:

NON-SUN Qlogic:

• qlaxxxx
• QLGC,qla
• QLGC,qlc
• SUNW,qla

NON-SUN Emulex:

• emlx (emlxs is SUN)
• lpfc
  
  Click on the below link for the detailed explanation.
  
  Identify HBA card
  
  So we have got some idea on HBA cards identification now lets go ahead and configure the persistent for emulex card. As i have previously mentioned that persistent binding is needed only for SUN non-branded emulex card.
  
  To configure persistent binding, there are 2 files needed to be modified.
  
  /kernel/drv/lpfc.conf
  /kernel/drv/sd.conf
  
  lpfc.conf - Kernel configuration file for lpfc driver modules
  sd.conf - kernel configuration file for sd driver module
  
  lpfc.conf file is where you configure the persistent binding and sd.conf is where you should add enough entry to allow OS to scan for more devices.
  
  For persistent binding you need to find the HBA card instance you can easily determine this by using lputil commands
  
  Here is an example
  
  bash-3.00# /usr/sbin/lpfc/lputil
  LightPulse Common Utility for Solaris/SPARC. Version 2.0a13 (1/3/2006).
  Copyright (c) 2005, Emulex Corporation
  
  Emulex Fibre Channel Host Adapters Detected: 3
  Host Adapter 0 (lpfc3) is an LP11002-E (Ready Mode)
  Host Adapter 1 (lpfc4) is an LP11002-E (Ready Mode)
  Host Adapter 2 (lpfc2) is an LP9802 (Ready Mode)
  
  
  As you can see above there are 3 cards. LP11002-E looks like a dual port card that can be confirmed by looking at /etc/path_to_inst file . Here is extract from the the file
  
  
  "/pci@1c,600000/lpfc@1,1" 1 "emlxs"
  "/pci@1c,600000/lpfc@1,1" 1 "lpfc"
  "/pci@1c,600000/lpfc@1,1/fp@0,0" 1 "fp"
  "/pci@1c,600000/fibre-channel@1" 3 "lpfc"
  "/pci@1c,600000/fibre-channel@1,1" 4 "lpfc"
  "/pci@1d,700000/lpfc@2" 2 "emlxs"
  "/pci@1d,700000/lpfc@2" 2 "lpfc"
  "/pci@1d,700000/lpfc@2/fp@0,0" 2 "fp"
  
  From the above 
•  
• "/pci@1c,600000/fibre-channel@1" 3 "lpfc"
  "/pci@1c,600000/fibre-channel@1,1" 4 "lpfc"
  
  
  See the physical path of the lpfc which is same with the instance as 3 and 4 so its a single HBA card having dual port so lpfc3 and lpfc4 is the instance of dual port HBA card.
  
  Sample sd.conf file:
  
  name="sd" parent="lpfc" target=0 lun=0 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=1 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=2 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=3 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=4 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=5 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=6 hba="lpfc0";
  name="sd" parent="lpfc" target=0 lun=7 hba="lpfc0";
  
  name="sd" parent="lpfc" target=1 lun=0 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=1 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=2 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=3 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=4 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=5 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=6 hba="lpfc1";
  name="sd" parent="lpfc" target=1 lun=7 hba="lpfc1";
  
  
  
  
  
  Now in the above output assigning lun no is a tricky one. You need to work with storage team for it. Get the HEX code for each LUN and convert it to decimal which needs to be specified as lun no.
  
  Another example of /etc/path_to_inst
  
  > grep -i lpfc /etc/path_to_inst
  "/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/fibre-channel@1" 0 "lpfc"
  "/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/fibre-channel@1,1" 1 "lpfc"
  "/pci@1e,600000/pci@0/pci@8/fibre-channel@0" 4 "lpfc"
  "/pci@1e,600000/pci@0/pci@8/fibre-channel@0,1" 5 "lpfc"
  "/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/fibre-channel@2" 2 "lpfc"
  "/pci@1f,700000/pci@0/pci@2/pci@0/pci@8/fibre-channel@2,1" 3 "lpfc"
  "/pci@1f,700000/pci@0/pci@9/fibre-channel@0" 6 "lpfc"
  "/pci@1f,700000/pci@0/pci@9/fibre-channel@0,1" 7 "lpfc"
  
  From the above you can find out that there 4 emulex dual ports HBA cards
  
  
  
  
  
  
  
  
  
  

Replacing a disk in Veritas Volume Manager

Replacing a disk in VxVM
--------------------------------------------------------

This has always been a difficult task while replacing a disk. The main worry which runs in our mind that data should not be lost while performing the activity, it should be intact otherwise you know the pain of facing other team.

Here i will be explaining very simple steps which will make you comfortable in doing such critical activities.

I will be talking about concatenated volume where you don't have redundant copies of data later i will cover the mirrored volume.

Disk replacement in concatenated volume

Here is the vxprint -htq output

bash-3.00# vxprint -htq
Disk group: appdg

dg appdg        default      default  4000     1347953587.80.vcs1

dm disk03       disk_3       auto     65536    2027168  -
dm disk04       disk_4       auto     65536    2027168  -

v  appvol       -            ENABLED  ACTIVE   3121152  SELECT    -        fsgen
pl appvol-01    appvol       ENABLED  ACTIVE   3121152  CONCAT    -        RW
sd disk03-01    appvol-01    disk03   0        2027168  0         disk_3   ENA
sd disk04-01    appvol-01    disk04   0        1093984  2027168   disk_4   ENA

In the above appdg diskgroup first subdisk is disk03-01 which is associated with disk03 and the plex is a concatenated type.

In the above scenario, if in case disk03 which is the first disk added to diskgroup and on which the Veritas has created the first subdisk for "appvol" volume. There is no way to replace the disk, if you try to do so, you will end up in corrupting the filesytem and the data.

But if you want to replace the second disk(flag - failing disk status)  which is absolutely possible without data loss but during this activity volume would not accessible. 

First we will use vxdiskadm to replace the disk, later we will try with only command prompt.

Connect the new disk in an empty slot and initialize it.

Run vxdiskadm from the command prompt, it will open up a list of menu items.


-> Select option no 4 - Remove a disk for replacement

Select the correct disk which you want to replace

(Caution - Do not select first the disk in the disgroup)

Once you have removed the disk it will prompt to select a disk with which you want to replace.  Select the disk which you have initialized.http://www.blogger.com/blogger.g?blogID=7967882172610062116#editor/target=post;postID=2988077713332921354

Once you are done with the above steps, you will find the plex is in disabled and recover steps

# vxprint -htqv
Disk group: appdg

v  appvol       -            DISABLED ACTIVE   3121152  SELECT    -        fsgen
pl appvol-01    appvol       DISABLED RECOVER  3121152  CONCAT    -        RW
sd disk03-01    appvol-01    disk03   0        2027168  0         disk_3   ENA
sd disk04-01    appvol-01    disk04   0        1093984  2027168   disk_5   ENA
#

You need to correct the above

Below sequence of commands needs to executed to fix the abovfe

#vxmend -g appdg fix stale appvol-01

#vxmend -g appdg fix clean appvol-01

#vxvol -g appdg start appvol

That's it you are done. Mount the volume and you are ready to use the volume.


Replacing a failed disk in mirrored volume

Here is my volume status

# vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mir01        disk_3       auto     65536    2027168  -
dm mir02        disk_4       auto     65536    2027168  -

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
pl appvolmir-02 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir02-01     appvolmir-02 mir02    0        1024000  0         disk_4   ENA

Where appvolmir is mirrored volume in mir diskgroup, having 2 plexes appvolmir-01 and appvolmir-02

Change the plex status to offline

#vxmend -g mir off appvolmir-02

# vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mir01        disk_3       auto     65536    2027168  -
dm mir02        disk_4       auto     65536    2027168  -

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
pl appvolmir-02 appvolmir    DISABLED OFFLINE  1024000  CONCAT    -        RW
sd mir02-01     appvolmir-02 mir02    0        1024000  0         disk_4   ENA
# 

Disassociates the offline plex from the volume

#vxplex -g mir dis  appvolmir-02

# vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mir01        disk_3       auto     65536    2027168  -
dm mir02        disk_4       auto     65536    2027168  -

pl appvolmir-02 -            DISABLED -        1024000  CONCAT    -        RW
sd mir02-01     appvolmir-0 mir02    0        1024000  0         disk_4   ENA

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
#
 
 Remove the plex

#vxedit -g mir -r rm appvolmir-02

 # vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mir01        disk_3       auto     65536    2027168  -
dm mir02        disk_4       auto     65536    2027168  -

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
#
Once plex is removed, removed the disk from veritas control

#vxdg -g mir rmdisk mir02 




# vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mir01        disk_3       auto     65536    2027168  -

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
#


So disk removal is done, Now its time to attach a new disk and sync the data. Identify a new disk should be of similar or more size.

Here in this example i would be using the disk_5 as a replacement disk.

# vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mirreplacedisk disk_5     auto     65536    41764864 -
dm mir01        disk_3       auto     65536    2027168  -

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
#

So once the new disk is initialized and added to the diskgroup , mirror the volume. Here is the vxdisk list ouput

# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0d0s2       auto:sliced     ibmdg01      ibmdg        online
disk_3       auto:cdsdisk    mir01        mir          online
disk_4       auto:cdsdisk    -            -            online
disk_5       auto:cdsdisk    mirreplacedisk  mir          online
disk_6       auto:SVM        -            -            SVM
disk_7       auto:SVM        -            -            SVM
disk_8       auto:ZFS        -            -            ZFS
disk_9       auto:ZFS        -            -            ZFS
disk_10      auto:ZFS        -            -            ZFS
disk_11      auto:cdsdisk    -            -            online
disk_12      auto:cdsdisk    -            -            online

"mirrreplacedisk" is the replaced disk

#vxassist -g mir make mirror appvolmir mirreplacedisk

# vxprint -htqg mir
dg mir          default      default  4000     1348198265.15.vcs1

dm mirreplacedisk disk_5     auto     65536    41764864 -
dm mir01        disk_3       auto     65536    2027168  -

v  appvolmir    -            ENABLED  ACTIVE   1024000  SELECT    -        fsgen
pl appvolmir-01 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mir01-01     appvolmir-01 mir01    0        1024000  0         disk_3   ENA
pl appvolmir-02 appvolmir    ENABLED  ACTIVE   1024000  CONCAT    -        RW
sd mirreplacedisk-01 appvolmir-02 mirreplacedisk 0 1024000 0      disk_5   ENA
#
 

Driver versus firmware

The difference of firmware and drivers is the application of this piece of software.
The most electronic hardware only needs firmware to run basicly. E.G. a DVD-player needs firmware to know how to read a CD or DVD. But to have it accessed under a operating system like Windows you need a drivhttp://www.blogger.com/blogger.g?blogID=7967882172610062116#editor/target=post;postID=5967573526606548086er to access the CD or DVD or watch a movie.
To visualize: A car needs an engine to drive at all, but needs a person (driver!)to control it

Firmware is the software that runs on the device. A driver is the software that tells your operating system how to communicate with the device. Not all devices have firmware--only devices with some level of intelligence.
Read more: http://wiki.answers.com/Q/What_is_the_difference_between_firmware_and_drivers#ixzz25ieOoLEf