User ManualTable of ContentsSun StorEdge™ 3900 and 6900 Series Troubleshooting Guide1Contents3List of Figures9Preface11How This Book Is Organized11Using UNIX Commands12Typographic Conventions13Shell Prompts13Related Documentation14Accessing Sun Documentation Online15Sun Welcomes Your Comments15Chapter 1. Introduction17Introduction17Predictive Failure Analysis Capabilities18Chapter 2. General Troubleshooting19Storage Automated Diagnostic Environment Event Grid37To Customize an Event Report37Fibre Channel Link Diagrams32FIGURE21 Sun StorEdge 3900 Series Fibre Channel Link Diagram32FIGURE22 Sun StorEdge 6900 Series Fibre Channel Link Diagram33Host Side Troubleshooting34Storage Service Processor Side Troubleshooting34Command Line Test Examples35qlctest(1M)35switchtest(1M)36Fibre Channel Links31Multipathing Options in the Sun StorEdge 6900 Series23Alternatives to Sun StorEdge Traffic Manager24To Quiesce the I/O241. Determine the path you want to disable.242. Type:24To Unconfigure the c2 Path241. Type:242. Using Storage Automated Diagnostic Environment Topology GUI, determine which virtualization en...253. Use the world wide name (WWN) of the virtualization engine that is in the unconfigure command,...254. Verify that I/O has halted.25To Suspend the I/O261. Stop all customer applications that are accessing the Sun StorEdge T3+ array.262. Manually pull the link from the Sun StorEdge T3+ array to the switch and wait for a Sun StorEd...26To Return the Path to Production261. Type cfgadm -c configure device.262. Verify that I/O has resumed on all paths.26To View the VxDisk Properties271. Type the following:272. Use the luxadm(1M) command to display further information about the underlying LUN.28To Quiesce the I/O on the A3/B3 Link291. Determine the path you want to disable.292. Disable the path by typing the following:293. Verify that the path is disabled:29To Suspend the I/O on the A3/B3 Link291. Stop all customer applications that are accessing the Sun StorEdge T3+ array.292. Manually pull the link from the Sun StorEdge T3+ array to the switch and wait for a Sun StorEd...29a. After the failover occurs, replace the cable and proceed with testing and FRU isolation.29b. After testing is complete and any FRU replacement is finished, return the controller state bac...29To Return the Path to Production301. Type:302. Verify that the path has been re-enabled by typing:30General Troubleshooting Procedures19Troubleshooting Overview Tasks191. Discover the error by checking one or more of the following messages or files:202. Determine the extent of the problem by using one or more of the following methods:203. Check the status of a Sun StorEdge T3+ array by using one or more of the following methods:204. Check the status of the Sun StorEdge FC network switch-8 and switch-16 switches using the foll...215. Check the status of the virtualization engine using one or more of the following methods:216. Quiesce the I/O along the path to be tested as follows:217. Test and isolate the FRUs using the following tools:218. Verify the fix using the following tools:229. Return the path to service by using one of the following methods:22Chapter 3. Troubleshooting the Fibre Channel Links39A1/B1 Fibre Channel (FC) Link39To Verify the Data Host41FRU Tests Available for A1/B1 FC Link Segment42To Isolate the A1/B1 FC Link44A2/B2 Fibre Channel (FC) Link45To Verify the Host Side47To Verify the A2/B2 FC Link49FRU Tests Available for A2/B2 FC Link Segment49To Isolate the A2/B2 FC Link49A3/B3 Fibre Channel (FC) Link51To Verify the Host Side53To Verify the Storage Service Processor54FRU Tests Available for the A3/B3 FC Link Segment54To Isolate the A3/B3 FC Link55A4/B4 Fibre Channel (FC) Link56To Verify the Data Host58FRU tests available for the A4/B4 FC Link Segment60To Isolate the A4/B4 FC Link60Chapter 4. Configuration Settings63Verifying Configuration Settings63Chapter 5. Troubleshooting Host Devices69Host Event Grid69Using the Host Event Grid69To Replace the Master Host73To Replace the Alternate Master or Slave Monitoring Host741. Choose Maintenance -> General Maintenance -> Maintain Hosts.742. In the Maintain Hosts window, select the host to be replaced from the Existing Hosts list, and...743. Install the new host.744. Install the SUNWstade package on the new host.745. Run /opt/SUNWstade/bin/ras_install.746. Configure the host as a slave.747. Choose Maintenance -> General Maintenance -> Maintain Hosts.758. In the Maintain Hosts window, select the new host.759. Configure the options as needed.7510. Choose Maintenance -> Topology Maintenance -> Topology Snapshot.75a. In the Topology Snapshot window, select the new host.75b. Click Create and Retrieve Selected Topologies.75c. Click Merge and Push Master Topology.75Conclusion75Troubleshooting Sun StorEdge FC Switch-8 and Switch-16 Devices77Chapter 6. Troubleshooting Sun StorEdge FC Switches77To Diagnose and Troubleshoot Switch Hardware78Switch Event Grid78Replacing the Master Midplane84Chapter 7. Troubleshooting Virtualization Engine Devices85Virtualization Engine Description85Virtualization Engine Diagnostics86Service Request Numbers86Service and Diagnostic Codes86To Retrieve Service Information86Virtualization Engine LEDs88Power LED Codes89Interpreting LED Service and Diagnostic Codes89Back Panel Features90Ethernet Port LEDs90Fibre Channel Link Error Status Report91To Check Fibre Channel Link Error Status Manually92Sun StorEdge 6900 Series Multipathing Example105Virtualization Engine Event Grid111Using the Virtualization Engine Event Grid111Troubleshooting the Sun StorEdge T3+ Array Devices115One Sun StorEdge T3+ array partner pair with 1 500GB RAID 5 LUN per brick (2 LUNs total)105FIGURE72 Sun StorEdge 6900 Series Logical View106FIGURE73 Primary Data Paths to the Alternate Master107FIGURE74 Primary Data Paths to the Master Sun StorEdge T3+ Array108FIGURE75 Path Failure—Before the Second Tier of Switches109FIGURE76 Path Failure —I/O Routed through Both HBAs110Chapter 8. Troubleshooting Sun StorEdge T3+ Array Devices115Troubleshooting the T1/T2 Data Path118T1/T2 Notification Events119Sun StorEdge T3+ Array Storage Service Processor Verification122T1/T2 FRU Tests Available123T1/T2 Isolation Procedures124Sun StorEdge T3+ Array Event Grid125Using the Sun StorEdge T3+ Array Event Grid1251. From the Storage Automated Diagnostic Environment Help menu, click the Event Grid link.1252. Select the criteria from the Storage Automated Diagnostic Environment event grid, like the one...125FIGURE85 Sun StorEdge T3+ array Event Grid1251. Voltage level on power supply and battery have moved out of acceptable thresholds.1272. The internal PCU temp has exceeded acceptable thresholds.1273. A PCU fan has failed.1271. Telnet to affected Sun StorEdge T3+ array1262. Verify disk state in fru stat, fru list, and vol stat.’1261. Telnet to affected Sun StorEdge T3+ array.1262. Verify tje loopcard state with fru stat.1263. Verify the matching firmware with the other loopcard.1264. Re-enable the loopcard if possible (enable u (encid)|[1|2] ). Replace loopcard if necessary.1265. Re-enable the disk if possible1266. Replace the disk, if necessary.1261. Telnet to the affected Sun StorEdge T3+ array.1272. Run refresh -s to verify the battery state.1273. Replace the battery, if necessary1271. Telnet to affected Sun StorEdge T3+ array.1272. Verify the fan state with fru stat.1273. Replace the power cooling unit, if necessary.1271. Telnet to affected Sun StorEdge T3+ array.1282. Verify power cooling unit state in fru stat.1283. Replace PCU, if necessary.1281. Telnet to the affected Sun StorEdge T3+ array.1282. Verify that the power cooling unit state is in ‘fru stat’1283. Replace the PCU if necessary.1281. Verify luxadm via command line (luxadm probe, luxadm display)1302. Verify cables, GBICs and connections along data path.1303. Check the Storage Automated Diagnostic Environment SAN Topology GUI to identify the failing se...1304. Verify the correct FC switch configuration, if applicable.1301. Check Ethernet connectivity to the affected Sun StorEdge T3+ array.1302. Verify the Sun StorEdge T3+ array is booted correctly.1303. Verify the correct TCP/IP settings on the Sun StorEdge T3+ array .1304. Increase the http and/or ping timeout in Utilities-- >System-- >System-- >Timeouts. The curren...1301. Telnet to affected Sun StorEdge T3+ array.1352. Verify the controller state with ‘fru stat’ and ‘sys stat’.1353. Run ‘logger - dmprstlog’ to capture controller information.1354. Re-enable the controller if possible (enable u)1355. Replace the controller, if necessary.1351. Telnet to the affected Sun StorEdge T3+ array1352. Verify that the disk state is in fru stat, fru list, and vol stat.1353. Replace the disk, if necessary.1351. Telnet to the affected Sun StorEdge T3+ array.1362. Verify loopcard state with fru stat1363. Verify matching firmware with other loopcard.1364. Re-enable loopcard if possible (enable u(encid)|[1|2|]1365. Replace the loopcard if necessary.1361. Check the Sun StorEdge T3+ array syslog for battery hold times.1372. If < 6 minutes, replace the battery, or the entire PCU, as required.1371. Telnet to the affected Sun StorEdge T3+ array1372. Check the status of LUNs via vol mode or vol stat.137Replacing the Master Midplane138Explorer Data Collection Utility115Chapter 9. Troubleshooting Ethernet Hubs139Appendix A-Virtualization Engine Reference141TABLEA1 SRN and SNMP Reference141TABLEA2 SRN/SNMP Single Point of Failure Table144TABLEA3 Port Communication145TABLEA4 Service Codes145Appendix B-SUNWsecfg Error Messages147TABLEB1 Virtualization Engine SUNWsecfg Error Messages148TABLEB2 Sun StorEdge Network FC Switch-8 and Switch-16 Switch SUNWsecfg Error Messages151TABLEB3 Sun StorEdge T3+ Array SUNWsecfg Error Messages153TABLEB4 Other SUNWsecfg Error Messages156setupswitch Exit Values157Index159Size: 1.15 MBPages: 162Language: EnglishOpen manual