Help with "Production" server

jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
I am currently having a network issue with one of our licensing "production" server, we've tried different steps to resolve the issue with no success.

Steps Taken:

- Replaced Dell Server MOBO
- Consulted w/switch Manufacturer Force 10 (No problem w/switch)
- Rolled Back Nic drivers /uninstall drivers
- Teamed and unteamed interfaces (Bridge)
- Changed Static IP address
- Used a different switch

Facts:

- Arp tables on client include all host on network segmnent, except gateway
- Statically assign a mac address to gateway does not help
- ARP tables on switch identify correct server interface
- Server can ping itself (10.10.0.70), but cannot ping other devices on local LAN

Thanks in advance!!!
«1

Comments

  • snadamsnadam Member Posts: 2,234 ■■■■□□□□□□
    jbaello wrote:
    I am currently having a network issue with one of our licensing "production" server, we've tried different steps to resolve the issue with no success.

    Steps Taken:

    - Replaced Dell Server MOBO
    - Consulted w/switch Manufacturer Force 10 (No problem w/switch)
    - Rolled Back Nic drivers /uninstall drivers
    - Teamed and unteamed interfaces (Bridge)
    - Changed Static IP address
    - Used a different switch

    Facts:

    - Arp tables on client include all host on network segmnent, except gateway
    - Statically assign a mac address to gateway does not help
    - ARP tables on switch identify correct server interface
    - Server can ping itself (10.10.0.70), but cannot ping other devices on local LAN

    Thanks in advance!!!

    Have you ran a sniffer on it yet to see where exactly the packets start dropping? just for fun, have you tried to configure it to have its address assigned via DHCP? Also have you thrown it on another network separate from the current one to see if the problems still arise?
    also, have you tried installing another NIC to see if the problem still continues (although its a stretch and highly unlikely).
    **** ARE FOR CHUMPS! Don't be a chump! Validate your material with certguard.com search engine

    :study: Current 2015 Goals: JNCIP-SEC JNCIS-ENT CCNA-Security
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    I went ahead and boot a Win PE and it looks like network is working perfectly, this is a no brainer solution I suggested from one of our Network Admin who's been stucked working with this network issue for the last two days, now hardware is finally eliminated as the culprit, by the look of it the OS has some major issue.

    Win PE network configurator is configured as a DHCP host, I'll try and set the OS network to DHCP since the Network Admin finally "thrown the towel" I have not used a sniffer yet at all...
  • snadamsnadam Member Posts: 2,234 ■■■■□□□□□□
    jbaello wrote:
    I went ahead and boot a Win PE and it looks like network is working perfectly, this is a no brainer solution I suggested from one of our Network Admin who's been stucked working with this network issue for the last two days, now hardware is finally eliminated as the culprit, by the look of it the OS has some major issue.

    ahh okay. omit my last comments then

    maybe TCP/IP is jacked up on this particular machine? have you tried uninstall/reinstall of tcp/ip?
    **** ARE FOR CHUMPS! Don't be a chump! Validate your material with certguard.com search engine

    :study: Current 2015 Goals: JNCIP-SEC JNCIS-ENT CCNA-Security
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    snadam wrote:
    jbaello wrote:
    I went ahead and boot a Win PE and it looks like network is working perfectly, this is a no brainer solution I suggested from one of our Network Admin who's been stucked working with this network issue for the last two days, now hardware is finally eliminated as the culprit, by the look of it the OS has some major issue.

    ahh okay. omit my last comments then

    maybe TCP/IP is jacked up on this particular machine? have you tried uninstall/reinstall of tcp/ip?

    Hmmm Yummy I'll get all the credits as soon this works, I'll try this... but isn't this the same, when you uninstall the NIC driver it basically blew out TCP/IP as well?
  • snadamsnadam Member Posts: 2,234 ■■■■□□□□□□
    jbaello wrote:
    snadam wrote:
    jbaello wrote:
    I went ahead and boot a Win PE and it looks like network is working perfectly, this is a no brainer solution I suggested from one of our Network Admin who's been stucked working with this network issue for the last two days, now hardware is finally eliminated as the culprit, by the look of it the OS has some major issue.

    ahh okay. omit my last comments then

    maybe TCP/IP is jacked up on this particular machine? have you tried uninstall/reinstall of tcp/ip?

    Hmmm Yummy I'll get all the credits as soon this works, I'll try this... but isn't this the same, when you uninstall the NIC driver it basically blew out TCP/IP as well?

    I was always under the impression that TCP/IP isntall/uninstall was OS related. Then you just bind the protocols to the NICS themselves

    check out this link from MS http://support.microsoft.com/kb/325356

    EDIT: the link is related to DC's...hang on and let me find a better one....http://support.microsoft.com/kb/317518

    you should be able to use netsh to 'reset' it
    **** ARE FOR CHUMPS! Don't be a chump! Validate your material with certguard.com search engine

    :study: Current 2015 Goals: JNCIP-SEC JNCIS-ENT CCNA-Security
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    No go... after running the netsh commands...
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    When I assign the server to be a DHCP host it was getting an APIPA, our DHCP server is working fine, and able to lease out address just normally, I am trying to disable all 3rd party services and see if this works, any networker out there that can give me more procedures let me know...
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    What OS/Service Pack is this? It definitely sounds like something in the IP stack is screwed up since layer 2 is fine, but you say you can ping yourself? I assume the NIC(s) are integrated on the motherboard (hence why you replaced it) but if you could confirm. How many NICs do you have in the server?
  • blargoeblargoe Member Posts: 4,174 ■■■■■■■■■□
    Is the network connection set to autonegotiate the link duplex and speed?
    IT guy since 12/00

    Recent: 11/2019 - RHCSA (RHEL 7); 2/2019 - Updated VCP to 6.5 (just a few days before VMware discontinued the re-cert policy...)
    Working on: RHCE/Ansible
    Future: Probably continued Red Hat Immersion, Possibly VCAP Design, or maybe a completely different path. Depends on job demands...
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    Started OS on "Safe Mode with Networking" network connection is working fine, it looks like the TCP/IP stack is intact, something else might be blocking network connectivity but which one? all services has already been disabled via "msconfig"
  • snadamsnadam Member Posts: 2,234 ■■■■□□□□□□
    astorrs wrote:
    What OS/Service Pack is this? It definitely sounds like something in the IP stack is screwed up since layer 2 is fine, but you say you can ping yourself? I assume the NIC(s) are integrated on the motherboard (hence why you replaced it) but if you could confirm. How many NICs do you have in the server?

    my guess form this is that its server 2k3, but im not sure if its a DC or not. If the NICS are integrated, then we can almost rule out bad hardware, because he said the MOBO was replaced. Perhaps it IS the actual NIC drivers causing the issue? Are the NICs showing up in device manager? I have a dell server with integrated nics, and everytime you restart, the drivers are not detected and i have to go in and manually 'update' them and they instantly find them. i know its kind of off target, but I figured Id ask.
    **** ARE FOR CHUMPS! Don't be a chump! Validate your material with certguard.com search engine

    :study: Current 2015 Goals: JNCIP-SEC JNCIS-ENT CCNA-Security
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    Server has two NIC and both is plugged/enabled to the switch...
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    Okay if it works in safe mode with networking then its not drivers. It would help to have confirmation on the O/S...

    Do you have any software firewalls enabled? I would disable every service not essential to booting the server, reboot and see if networking works.
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    astorrs wrote:
    Okay if it works in safe mode with networking then its not drivers. It would help to have confirmation on the O/S...

    Do you have any software firewalls enabled? I would disable every service not essential to booting the server, reboot and see if networking works.

    all services has already been disabled via "msconfig"

    I tried "netsh winsock reset" - about to see the result...
  • snadamsnadam Member Posts: 2,234 ■■■■□□□□□□
    astorrs wrote:
    Okay if it works in safe mode with networking then its not drivers. It would help to have confirmation on the O/S...

    Do you have any software firewalls enabled? I would disable every service not essential to booting the server, reboot and see if networking works.

    damn! I thought we were getting somewhere! I agree with astorrs, now that we have that bit o' info that its working in safe mode. Yay, trial and error time!!! :)
    EDIT: man i am slow tonight, let us know how that works out.
    **** ARE FOR CHUMPS! Don't be a chump! Validate your material with certguard.com search engine

    :study: Current 2015 Goals: JNCIP-SEC JNCIS-ENT CCNA-Security
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    winsock reset is a no go... I will try to modify registry for winsock entry and delete it...

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Winsock
    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Winsock2
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    Are there any other networking protocols installed on the machine?
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    Windows Server R2 Standard Edition Ver. 5.2 R2 (Build 3790 etc...) SP1
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    astorrs wrote:
    Are there any other networking protocols installed on the machine?
    no it was bridged at the beginning but I blow out bridging and deleted bridge protocol as well on network connection.
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    Can you double-check that the msconfig actually disabled all the additional services? (I've seen it miss some before because the vendor decided to mark them as system/boot).
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    I'm contemplating about this, but would not blow out the server, since it's a high critical production box, I will try to install a standalone NIC and see d outcome, but for now I'm going to call it a day, I've already done a lot of mistake :P well the network admin did... lol

    http://support.microsoft.com/kb/325356
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    astorrs wrote:
    Can you double-check that the msconfig actually disabled all the additional services? (I've seen it miss some before because the vendor decided to mark them as system/boot).

    All 3rd party services is infact disabled and confirmed, on regular mode DHCP is now able to obtain an IP address but still network connectivity is not working.
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    Wait let me understand this... you are getting a DHCP lease now? Even though you can't actually reach any other resources on the subnet.
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    Can you paste the output from a "route print" and an "ipconfig /all"
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    it's a very weird error, I think this is the 1st time I've encountered it, we are contemplating on just doing a setup/repair of windows installation, but first we are thinking of backing up everything...
  • jbaellojbaello Member Posts: 1,191 ■■■□□□□□□□
    Microsoft Windows [Version 5.2.3790]
    (C) Copyright 1985-2003 Microsoft Corp.

    C:\Documents and Settings\Administrator>route print

    IPv4 Route Table
    ===========================================================================
    Interface List
    0x1 ........................... MS TCP Loopback interface
    0x2 ...00 15 c5 ea 4c b9 ...... Broadcom BCM5708C NetXtreme II GigE (NDIS VBD
    ient) #2
    0x3 ...00 15 c5 ea 4c bb ...... Broadcom BCM5708C NetXtreme II GigE (NDIS VBD
    ient)
    ===========================================================================
    ===========================================================================
    Active Routes:
    Network Destination Netmask Gateway Interface Metric
    0.0.0.0 0.0.0.0 10.10.0.65 10.10.0.124 10
    0.0.0.0 0.0.0.0 10.10.0.65 10.10.0.123 10
    10.10.0.64 255.255.255.192 10.10.0.123 10.10.0.123 10
    10.10.0.64 255.255.255.192 10.10.0.124 10.10.0.124 10
    10.10.0.123 255.255.255.255 127.0.0.1 127.0.0.1 10
    10.10.0.124 255.255.255.255 127.0.0.1 127.0.0.1 10
    10.255.255.255 255.255.255.255 10.10.0.123 10.10.0.123 10
    10.255.255.255 255.255.255.255 10.10.0.124 10.10.0.124 10
    127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
    224.0.0.0 240.0.0.0 10.10.0.123 10.10.0.123 10
    224.0.0.0 240.0.0.0 10.10.0.124 10.10.0.124 10
    255.255.255.255 255.255.255.255 10.10.0.123 10.10.0.123 1
    255.255.255.255 255.255.255.255 10.10.0.124 10.10.0.124 1
    Default Gateway: 10.10.0.65
    ===========================================================================
    Persistent Routes:
    None

    C:\Documents and Settings\Administrator>
    C:\Documents and Settings\Administrator>ipconfig /all

    Windows IP Configuration

    Host Name . . . . . . . . . . . . : apu
    Primary Dns Suffix . . . . . . . : corp.rttusa.com
    Node Type . . . . . . . . . . . . : Unknown
    IP Routing Enabled. . . . . . . . : No
    WINS Proxy Enabled. . . . . . . . : No
    DNS Suffix Search List. . . . . . : corp.rttusa.com
    rttusa.com

    Ethernet adapter Local Area Connection 2:

    Connection-specific DNS Suffix . : corp.rttusa.com
    Description . . . . . . . . . . . : Broadcom BCM5708C NetXtreme II GigE (N
    VBD Client) #2
    Physical Address. . . . . . . . . : 00-15-C5-EA-4C-B9
    DHCP Enabled. . . . . . . . . . . : Yes
    Autoconfiguration Enabled . . . . : Yes
    IP Address. . . . . . . . . . . . : 10.10.0.123
    Subnet Mask . . . . . . . . . . . : 255.255.255.192
    Default Gateway . . . . . . . . . : 10.10.0.65
    DHCP Server . . . . . . . . . . . : 10.10.0.66
    DNS Servers . . . . . . . . . . . : 10.10.0.66
    10.10.0.67
    Lease Obtained. . . . . . . . . . : Wednesday, May 28, 2008 7:30:44 PM
    Lease Expires . . . . . . . . . . : Thursday, June 05, 2008 7:30:44 PM

    Ethernet adapter Local Area Connection:

    Connection-specific DNS Suffix . : corp.rttusa.com
    Description . . . . . . . . . . . : Broadcom BCM5708C NetXtreme II GigE (N
    VBD Client)
    Physical Address. . . . . . . . . : 00-15-C5-EA-4C-BB
    DHCP Enabled. . . . . . . . . . . : Yes
    Autoconfiguration Enabled . . . . : Yes
    IP Address. . . . . . . . . . . . : 10.10.0.124
    Subnet Mask . . . . . . . . . . . : 255.255.255.192
    Default Gateway . . . . . . . . . : 10.10.0.65
    DHCP Server . . . . . . . . . . . : 10.10.0.66
    DNS Servers . . . . . . . . . . . : 10.10.0.66
    10.10.0.67
    Lease Obtained. . . . . . . . . . : Wednesday, May 28, 2008 7:30:44 PM
    Lease Expires . . . . . . . . . . : Thursday, June 05, 2008 7:30:44 PM

    C:\Documents and Settings\Administrator> have fun!!!
  • AhriakinAhriakin Member Posts: 1,799 ■■■■■■■■□□
    Is the Windows Firewall/ICS Service still off? Have you checked the TCP/IP Filter section on the RCP/IP-Advanced-Options properties page?
    Have you scanned for rootkits in Safe Mode + Networking?
    Check your IPSEC Policies and make sure the server is not set to require it for all connections.
    We responded to the Year 2000 issue with "Y2K" solutions...isn't this the kind of thinking that got us into trouble in the first place?
  • astorrsastorrs Member Posts: 3,139 ■■■■■■□□□□
    Ahriakin wrote:
    Is the Windows Firewall/ICS Service still off? Have you checked the TCP/IP Filter section on the RCP/IP-Advanced-Options properties page?
    Have you scanned for rootkits in Safe Mode + Networking?
    Check your IPSEC Policies and make sure the server is not set to require it for all connections.

    And disconnect one of the two NICs to simplify things.
  • snadamsnadam Member Posts: 2,234 ■■■■□□□□□□
    jbaello wrote:
    0x2 ...00 15 c5 ea 4c b9 ...... Broadcom BCM5708C NetXtreme II GigE (NDIS VBD
    ient) #2
    0x3 ...00 15 c5 ea 4c bb ...... Broadcom BCM5708C NetXtreme II GigE (NDIS VBD
    ient)

    yep, have the same NICs in one of my power edge's. Unfortunately I dont have the same problem as you, but at least you can feel a little bit better that they are causing anguish for at least another person icon_wink.gif


    keep us posted tomorrow.
    **** ARE FOR CHUMPS! Don't be a chump! Validate your material with certguard.com search engine

    :study: Current 2015 Goals: JNCIP-SEC JNCIS-ENT CCNA-Security
Sign In or Register to comment.