Ladies and Gentlemen of the SBS/Technet forums,
I have here a serious problem that has been occurring over the past several months but, like a phantom, it comes and goes and I have never figured it out. I have gotten closer lately with a full debugger output for you to help me with if you have a moment.
This server randomly reboots, sometimes twice a day, sometimes it goes for months with no issue. I have captured some dumps from awhile back, but haven't gotten a most recent dump but hope to generate one with the Control+scroll lock(x2) combo soon.
I apologise in advance if the output is wrong or formatted poorly, I don't usually get this deep in BSOD analysis, but am very interested in figuring this out. I have several other dump files but they are all close in time and all due to the same issue. Here
are some files in a dropbox folder for you (the debugger output is from the April 14 dump) : https://www.dropbox.com/sh/w6y3oqt9upfsoot/M59qmOrJos
System info:SBS2011 Standard HP Proliant ML 150G6
3 Nics but one is disabled, one active in-use nic, and one ILO nic for remote mgmt.
edit 1 : the nic is updated to the latest driver...Intel Pro/1000 GT Desktop Adapter
Driver version 8.4.1.0 5/28/2008 (seems REALLY old but cant find updates anywhere)
I will paste the debugger output here:
Microsoft (R) Windows Debugger Version 6.2.9200.20512 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [J:\041413-33961-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available
Symbol search path is: srv*C:\websymbols*http://msdl.microsoft.com/download/symbols
Executable search path is: C:\Windows
Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: LanManNt, suite: SmallBusiness TerminalServer SmallBusinessRestricted SingleUserTS
Built by: 7601.18044.amd64fre.win7sp1_gdr.130104-1431
Machine Name:
Kernel base = 0xfffff800`01a68000 PsLoadedModuleList = 0xfffff800`01cac670
Debug session time: Sun Apr 14 23:02:27.476 2013 (UTC - 7:00)
System Uptime: 0 days 23:53:00.803
Loading Kernel Symbols
...............................................................
................................................................
.....................
Loading User Symbols
Loading unloaded module list
..........
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck D1, {0, 2, 0, fffff88001892490}
Probably caused by : NETIO.SYS ( NETIO!NetioDereferenceNetBufferList+86 )
Followup: MachineOwner
---------
4: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff88001892490, address which referenced memory
Debugging Details:
------------------
READ_ADDRESS: GetPointerFromAddress: unable to read from fffff80001d16100
GetUlongFromAddress: unable to read from fffff80001d161c0
0000000000000000 Nonpaged pool
CURRENT_IRQL: 2
FAULTING_IP:
tcpip! ?? ::FNODOBFM::`string'+5a24
fffff880`01892490 488b01 mov rax,qword ptr [rcx]
CUSTOMER_CRASH_COUNT: 1
DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT_SERVER
BUGCHECK_STR: 0xD1
PROCESS_NAME: System
TAG_NOT_DEFINED_c000000f: FFFFF88001F82FB0
TRAP_FRAME: fffff88001f7b040 -- (.trap 0xfffff88001f7b040)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffffa802bef68d0 rbx=0000000000000000 rcx=0000000000000000
rdx=fffff88001f53180 rsi=0000000000000000 rdi=0000000000000000
rip=fffff88001892490 rsp=fffff88001f7b1d0 rbp=0000000000000000
r8=0000000000006888 r9=00000000000000d0 r10=fffff80001a68000
r11=000000000000029d r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
tcpip! ?? ::FNODOBFM::`string'+0x5a24:
fffff880`01892490 488b01 mov rax,qword ptr [rcx] ds:00000000`00000000=????????????????
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff80001add1e9 to fffff80001addc40
STACK_TEXT:
fffff880`01f7aef8 fffff800`01add1e9 : 00000000`0000000a 00000000`00000000 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
fffff880`01f7af00 fffff800`01adbe60 : fffff880`01f7b038 00000000`00000000 fffff880`01f7b048 fffffa80`13a99e20 : nt!KiBugCheckDispatch+0x69
fffff880`01f7b040 fffff880`01892490 : fffffa80`13a99e20 fffff880`01f53100 fffffa80`206c644d fffff800`01a26b7f : nt!KiPageFault+0x260
fffff880`01f7b1d0 fffff880`00e03316 : fffffa80`13a99e20 00000000`12df2c58 00000000`00000000 00000000`0000000f : tcpip! ?? ::FNODOBFM::`string'+0x5a24
fffff880`01f7b220 fffff880`00e02a72 : fffffa80`14a813b0 00000000`00000000 00000000`00000002 00000000`00001000 : NETIO!NetioDereferenceNetBufferList+0x86
fffff880`01f7b250 fffff880`01849f26 : fffffa80`14a813b0 00000000`00000000 fffff880`01f7b300 fffffa80`164bd030 : NETIO!NetioDereferenceNetBufferListChain+0x332
fffff880`01f7b320 fffff880`01821c54 : fffffa80`14f53250 fffffa80`157627d0 fffffa80`13174b40 fffffa80`13265030 : tcpip!IppCompleteAndFreePacketList+0xc6
fffff880`01f7b350 fffff880`01829fb8 : 00000000`00000000 fffffa80`13174b40 fffffa80`13265030 fffff880`0196e9a0 : tcpip!IppCleanupMfe+0x44
fffff880`01f7b380 fffff880`018885fc : 00000000`00000002 fffff880`01972ee0 fffff880`01f7b430 fffffa80`13174b40 : tcpip!IppDereferenceMfe+0x18
fffff880`01f7b3b0 fffff880`01887d1a : fffff880`01f7b600 00000000`00000000 fffffa80`13174b58 fffff880`01f7b600 : tcpip!IppMfeSetTimeOut+0x10c
fffff880`01f7b4d0 fffff880`0188848a : fffffa80`13c89a00 fffff880`01f7b600 00000000`00000000 fffff800`01ae332a : tcpip!IppCompartmentSetTimeout+0x9a
fffff880`01f7b540 fffff800`01ae894c : fffff880`00000004 00000000`000000f0 00000000`40841200 00000000`00000000 : tcpip!IppTimeout+0x5a
fffff880`01f7b570 fffff800`01ae87e6 : fffffa80`2ee0d168 fffffa80`2ee0d168 00000000`00000000 00000000`00000000 : nt!KiProcessTimerDpcTable+0x6c
fffff880`01f7b5e0 fffff800`01ae86ce : 000000c8`308d64f7 fffff880`01f7bc58 00000000`00541982 fffff880`01f565c8 : nt!KiProcessExpiredTimerList+0xc6
fffff880`01f7bc30 fffff800`01ae84b7 : 00000027`14d7eacb 00000027`00541982 00000027`14d7ea16 00000000`00000082 : nt!KiTimerExpiration+0x1be
fffff880`01f7bcd0 fffff800`01ad594a : fffff880`01f53180 fffff880`01f5e0c0 00000000`00000000 fffff880`012d0400 : nt!KiRetireDpcList+0x277
fffff880`01f7bd80 00000000`00000000 : fffff880`01f7c000 fffff880`01f76000 fffff880`01f7bd40 00000000`00000000 : nt!KiIdleLoop+0x5a
STACK_COMMAND: kb
FOLLOWUP_IP:
NETIO!NetioDereferenceNetBufferList+86
fffff880`00e03316 4885ff test rdi,rdi
SYMBOL_STACK_INDEX: 4
SYMBOL_NAME: NETIO!NetioDereferenceNetBufferList+86
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: NETIO
IMAGE_NAME: NETIO.SYS
DEBUG_FLR_IMAGE_TIMESTAMP: 5034f6a0
FAILURE_BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferList+86
BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferList+86
Followup: MachineOwner
---------
4: kd> lmvm NETIO
start end module name
fffff880`00e00000 fffff880`00e60000 NETIO (pdb symbols) c:\websymbols\netio.pdb\76E292820E9A4B67A41B1C5BDB269DEE2\netio.pdb
Loaded symbol image file: NETIO.SYS
Mapped memory image file: c:\windows\System32\drivers\netio.sys
Image path: \SystemRoot\system32\DRIVERS\NETIO.SYS
Image name: NETIO.SYS
Timestamp: Wed Aug 22 08:11:28 2012 (5034F6A0)
CheckSum: 00067D76
ImageSize: 00060000
File version: 6.1.7601.17939
Product version: 6.1.7601.17939
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 3.6 Driver
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Windows® Operating System
InternalName: netio.sys
OriginalFilename: netio.sys
ProductVersion: 6.1.7601.17939
FileVersion: 6.1.7601.17939 (win7sp1_gdr.120822-0331)
FileDescription: Network I/O Subsystem
LegalCopyright: © Microsoft Corporation. All rights reserved.
Thank you friends! I really appreciate the help!