iSCSi issues?

higherho · June 2013

Hello all,

I have an interesting issue. I have a separate ESXi cluster residing in a 172.XX.XX.XXX subnet on a different switch and my datastore's reside in a 10.XX.XX.XX subnet (and different VLAN) on another switch. I know for best practices that I should't route ISICI traffic but I wanted to test some stuff out. I setup both switches the ability to route (RIPv2) and everything was fine and then one one day the links drop on the tester cluster. No errors, no dead link errors, nothing. I'm going to inspect the logs on the ESXi host itself to see if anything odd is popping up. My Storage VMkernals are set correctly and I can ping everything just fine from both switches.

I rescanned for new datas stores / HBAs and even rebooted the host (just to rule that out) and still nothing. I dug a little deeper and found out that my network adapters on the tester cluster state they are observing the 169 IP range which I found very odd. I'm still troubleshooting this issue (and looking at the ESXi hosts logs itself) but I'm curious of anyone here has any other suggestions? maybe I'm overlooking something?

Thanks.

higherho · June 2013

So the errors I see in the ESXi observation log

"Login to iSCSi traget "target name" on vmhba33 @ vmk2 failed The iSCSi initiator could not establish a network connection to target"

When I saw this I SSHed into the ESXi host and did a vmkping to all my SANS and I was able to hit everyone. I went to the SAN and configured it accordingly. Assigned a server (used my ESXi hsot IP) and I used the ISCSI name of the software adapter on the host for the inititaor.

their is another statment stating that my port vmk2 is not projected by the ESXi firewall but the configurations are set to default. The targets showed up before so I'm thinking is this a ESXi host issue or network issue.

Currently I am stumped...

meadIT · June 2013

Since you don't have an iSCSI vmkernel on the same VLAN your target, your host could be using any of the vmkernel interfaces when doing a vmkping. Try the pings again using the command below, but specifying the vmkernel being used by your iSCSI bindings.

esxcli network diag ping -I vmk2 -H 10.x.x.x

higherho · June 2013

Hi Mead,

Thanks for the assistance, I greatly appreciate it. I sshed into the ESXi host in question (172.3.4.70) and performed the following

esxcli network diag ping -I vmk2 -H 10.0.0.200

Summary:
Duplicated: 0
Host Addr: 10.0.0.200
Packet Lost: 0
Recieved: 3
Roundtrip Avg MS: 234
Roundtrip Max MS: 245
Roundtrip Min MS: 225
Transmitted: 3

Trace:
Detail:
Dup: false
Host: 10.0.0.200
ICMPSeq: 0
Received Bytes: 64
Roundtrip Time MS: 225
TTL: 62

Detail:
Dup: false
Host: 10.0.0.200
ICMPSeq: 1
Received Bytes: 64
Roundtrip Time MS: 233
TTL: 62

Detail:
Dup: false
Host: 10.0.0.200
ICMPSeq: 2
Received Bytes: 64
Roundtrip Time MS: 246
TTL: 62

what I dont understand is the subnets in the routing table on the switch can talk to each other. My port groups (VMkernal) are configured as follows;

Network label name
VLAN ID (none)
vMotion - enabled
iSCSI port binding Enabled

IP settings

172.3.4.16
255.255.255.0
172.3.4.1

CPD shows me my physical switch ports which I do have those two assigned to VLAN 100 which is 172.3.4.1 and a route in my routing table allowing traffic over the routed port to the 10.0.0.0 network. The physical network adapters on the vmnic are stating a non routeable IP in the observed IP ranges.

higherho · June 2013

Alright, well when I removed the VMkernel port bindings from the iSCSI initiator all of the Datastores came back. So kinda norrowed down were the issue is.

undomiel · June 2013

This article here describes what you were running into: VMware KB: Cannot reach iSCSI target after enabling iSCSI port binding on ESX/ESXi 4.x and ESXi 5.0

higherho · June 2013

Thanks Undomiel for the information but the hyper link is taking me to the vmware store.

meadIT · June 2013

Here's the important part for you from the link undomiel posted:

iSCSI port binding in ESX/ESXi 4.x and ESXi 5.x does not support routing.

To resolve this issue and still use iSCSI port binding, you must place the iSCSI initiator and the iSCSI target on the same subnet.

Alternatively, this can be resolved by disabling iSCSI port binding.

undomiel · June 2013

That's odd. I guess something must be getting stripped out of the URL somewhere. Do a search on the title in Google and you'll find it, Cannot reach iSCSI target after enabling iSCSI port binding on ESX/ESXi

higherho · June 2013

Wow that is a bummer, o well better to know now than later

Thank you all for the help / assistance! I greatly appreciate it.

iSCSi issues?

Comments