Service Unavailable Errors in IIS 7 Are Killing Me

UPDATE: Problem solved thanks to some members of the IIS 7 team!

I am at my wits end trying to get IIS 7 to work on my Vista Ultimate box and I have tried everything I can think of.

I’ve tried every step in the following tutorial, Where did my IIS7 Server go? Troubleshooting “server not found” errors. I also tried every step in this post on troubleshooting “service unavailable” errors. Trust me when I say I went through every one of these steps twice. Rob Conery can back me up on this because he watched me do so in a when I shared my desktop with him via GoToMeeting.

As far as I can tell, IIS 7 or http.sys must be corrupted somehow and the only thing left for me to try is to repave my machine and reinstall Vista. Unless of course one of my dear readers has an insight that will help me solve this, or knows someone who does.

The Problem

I’m running Vista Ultimate which has IIS 7 installed. When I navigate to http://localhost/ or http://localhost/iisstart.htm, I get an HTTP Error 503 Service Unavailable message.

What I’ve Tried

Confirmed that Skype is not listening to port 80 (Skype tries to listen on port 80 by default).

1. Confirmed that App Pools were configured correctly and started.

2. Ran the following command: appcmd list apppools which produced the output:

APPPOOL "DefaultAppPool" (MgdVersion:v2.0,MgdMode:Integrated,state:Started)
APPPOOL "Classic .NET AppPool" (MgdVersion:v2.0,MgdMode:Classic,state:Started)

3. Confirmed that Website was started.

4. Ran the following command: netstat -a -o -b which produced the output:

TCP [::]:80 METAVERSE:0 LISTENING 4
Can not obtain ownership information

The 4 there is the PID which I confirmed to be System as in the NT Kernal & System.

5. Confirmed bindings were configured in IIS Manager and set for localhost port 80.

6. No error messages in the event log under System nor Application.

7. Made sure that my user account and the Network Service account both have access to the c:\inetpub\wwwroot directory.

8. Tried browsing to http://MYCOMPUTERNAME/, http://localhost:80/, http://127.0.0.1/, etc... (I was getting desparate).

9. Tried changing the default AppPool’s managed pipeline mode to Classic.

10. Tried changing the default AppPool’s .NET framework version to No Managed Code (recall that I am trying to request a static HTML page).

11. Able to ping localhost.

12. Tried to telnet localhost 80 and then issue the command GET / and received the same message.

13. Double checked that all the Handler Mappings were enabled.

14. Made sure Anonymous Authentication was enabled. Heck, I tried it with them all enabled and tried it with only Windows Authentication enabled.

15. Authorization Rules has one rule: Mode=allow, Users=All Users.

16. Enabled Failed Request Tracking. Nothing showed up in the logs.

17. Uninstalled and reinstalled IIS 7.

18. Tried pulling my hair out and rending my garments and sacrificing chickens1.

Any ideas?

So there you go. I’ve tried everything I can think of and now I appeal to you for help.

The funny thing is that this works on my Vista box at work and I compared every setting in IIS. This confirms in my mind that something got fubar’d. But I hesitate to repave my machine just yet in the hope that someone out there has some definitive answers for me.

Technorati tags: , ,

1 No actual chickens were harmed in troubleshooting this problem.

What others have said

Requesting Gravatar... Damien Guard May 19, 2007 2:47 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Try using "telnet localhost 80" followed by "GET /" hitting return and seeing what comes back.

[)amien
Requesting Gravatar... Florian Krüsch May 19, 2007 4:37 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Try http://127.0.0.1 instead of http://128.0.0.1/ ?
Requesting Gravatar... Bill Staples May 19, 2007 7:07 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Hi - try this: go to \windows\system32\inetsrv\ and run the following command:

appcmd start DefaultAppPool

If that doesn't work, shoot me an email and I'll help you debug it further.

Bill
Requesting Gravatar... Geri Langlois May 19, 2007 7:08 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
I had a similar situation but no where near identical -- I only post this because it might give you some ideas of where to look.

I was setting up an asp 1.1 site with an asp 2.0 app below it. I created a 1.1 app pool for the main site and applied it. After that I started getting the service unavailable error.

In the end, after much hair pulling, I installed 1.1 SP1 on my machine and applied the app pool that it automatically created. It appears that the 1.1 framework installed with vista (ultimate) was out of sync with or unable to support IIS 7. The point is this is that it was a bad (incorrect) app pool that was the source of this issue and I wonder if that is what you should look at next. Hope this was helpful.

Requesting Gravatar... Scott May 19, 2007 7:16 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Have you tried Spy++ or the equivalent Sysinternals process spy program to see what's really going on in process 4?

You could also try writing a small program that will attempt to listen to port 80 and rule out any freaky firewall problems or port conflicts. Or download Apache or some small web server and see if it can bind to port 80.

Also you didn't mention looking at the event logs or IIS logs. I assume that you did, but what did they say? Did the IIS logs record the GET attempts? Does the system event log show anything funny? Isn't there some kind of "verbose" logging mode for IIS? Here's a module for IIS 7 logging
http://blogs.iis.net/tobintitus/archive/2007/01/30/logging-ui-for-iis-7-0-on-vista.aspx
Requesting Gravatar... Scott May 19, 2007 7:24 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Oh more dumb troubleshooting tips.

Can you serve up static HTTP pages on your box?

Checked the permissions on wwwroot?
Requesting Gravatar... Haacked May 19, 2007 8:14 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
THanks Scott. I did check NTFS permissions. I am also attempting to load a static html page with no luck.

I also created a brand new website with a static html page and couldn't load that.
Requesting Gravatar... Ayende Rahien May 19, 2007 9:27 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Phil,
The chicken thing is the problem.
You really need to sacrifice at least one to honor IIS.
Requesting Gravatar... Eber Irigoyen May 19, 2007 9:30 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
1 - tcpview
2 - fiddler
Requesting Gravatar... Tony Bunce May 19, 2007 9:48 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
This sounds like an App Pool issue to me. I have see it before when the asp.net user didn't have a profile. Running the bat file on this page has fixed it for me in the past:
http://support.microsoft.com/kb/827641

It says its only for asp.net 1.0 but I know it has fixed 1.1 problems for me before and maybe even 2.0 issues.

You could also try aspnet_regiis -ua to uninstall asp.net just to see if that is the problem.

The "Service Unavailable" is a page generated by IIS so i don't think any network level tools are going to help much.

Is UAC enabled? Maybe try turning it off. You could also try enabling security auditing to see if some security issue is causing the problem.

I think IIS is supposed to log something to event viewer when a worker process crashes, so make sure your event log isn't full and if it is clear it out or make it bigger.
Requesting Gravatar... engtech May 19, 2007 9:49 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Have you tried Apache?

:)
Requesting Gravatar... Haacked May 19, 2007 9:50 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@Damien - Still get the error.
Requesting Gravatar... Haacked May 19, 2007 10:08 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@engtech, just like your blog. Funny content, but not so helpful. ;)

I kid! I kid!
Requesting Gravatar... Mike Volodarsky May 19, 2007 12:57 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
I just made a post about debugging start-up crashes that may be helpful in getting to the bottom of your problem.

Check it out here: http://mvolo.com/blogs/serverside/archive/2007/05/19/Troubleshooting-IIS7-503-_2200_Service-unavailable_2200_-errors-with-startup-debugging.aspx.

I am a bit suprised that you didnt see an event in the Windows "Application" event log, that indicates that the worker process has suffered an AV (or 5 of these, with a corresponding event in the System event log indicating that the application pool has entered rapid fail protection). Can you double-check whether these events are there?

Let me know if the technique above does not work for you for some reason, or what you find out - feel free to leave comments on the post.

Good luck,

Mike
Requesting Gravatar... Rick Strahl May 19, 2007 1:18 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Phil... Makes sure your AppPool that you have assigned to the app is actually running. In IIS 7 the AppPools can be stopped explicitly and that may get you the 503 error - I had that pop up on me a while back when the AppPool was shut down due too many failures.

http://west-wind.com/WebLog/posts/9436.aspx
Requesting Gravatar... Eric Kemp May 19, 2007 3:12 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Phil,

You might want to take a look in:

C:\Windows\System32\inetsrv\config

In that folder, you'll find applicationHost.config and applicationHost.config.clean.install

These effectively represent the replacement for the MetaBase in IIS 6 and earlier. It might be worth doing a diff on the two files, or simply replacing the existing applicationHost.config with a copy of applicationHost.config.clean.install.
Requesting Gravatar... Haacked May 19, 2007 4:18 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@Rick - I saw your post on that and I did make sure. If you take a look at step #2, you can see that both appPools are started. I also confirmed within IIS Manager. I also stopped and restarted both.

@Eric - I see the following two files:
- administration.config
- applicationHost.config

I don't see any files with the *.clean.install extension.
Requesting Gravatar... Haacked May 19, 2007 4:21 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@Mike - Rapid Fail Protection would cause the appPools to stop, right?

The appPools aren't stopped. I just checked again and they are still started. Again, I ran the command in step #2 above.
Requesting Gravatar... Eric Kemp May 19, 2007 4:56 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Hmm... the .clean.install files must not be created in all scenarios... However, I have both of them in two different Vista scenarios. Is this a new install, or an upgrade from XP?

In any event, the idea is to get back to default settings. The .config file may linger even after an uninstall of IIS. It would be worth trying the uninstall and reinstall again, making sure that you clean out any residual settings, such as applicationHost.config, before reinstalling.
Requesting Gravatar... Mike Volodarsky May 19, 2007 6:03 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Phil,

Can you answer the following questions for me?

1) After making the request, do you see any processes called "w3wp.exe" in your task manager (be sure to show all proceses in all sessions)

2) What is in your Windows "Application" event log? Are you seeing any events indicating why your worker processes are going away? I have never seen a case where an event is not logged.

3) Have you followed the steps in http://mvolo.com/blogs/serverside/archive/2007/05/19/Troubleshooting-IIS7-503-_2200_Service-unavailable_2200_-errors-with-startup-debugging.aspx?

Thanks,

Mike
Requesting Gravatar... Scott Elkin May 19, 2007 9:51 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
aspnet_regiis?
Requesting Gravatar... Haacked May 19, 2007 10:14 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@Mike

1. I don't see w3wp.exe in my task manager. That's interesting.

2. There's nothing in my application log related to this.

3. I haven't tried downloading the windows debug symbol. I did share my desktop via GoToMeeting with a member of the IIS7 team who was as stymied as I was. We'll open a trouble ticket.

If I have time tomorrow, I can run through all the steps in your post. I just wasn't sure if it was applicable given that there were no event log messages and the appPools are all started and never die.

When I run appcmd list apppools I always get the same result, showing them started.
Requesting Gravatar... jayson knight May 19, 2007 11:26 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
I can't be of much help other than to say that if you can't even get an HTML page to render then something is seriously wrong w/ the core of IIS. Did you check all your mappings/verb handlers at the root of IIS (above the web apps)? What about the IP inclusion list (blocking all IP addresses somehow)? Also, a more verbose reason should be listed in the http.sys error logs.
Requesting Gravatar... engtech May 20, 2007 3:47 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Wish I could be more helpful but I really don't know anything about using Microsoft development tools. I've been on a strict died of linux / open source software for years.
Requesting Gravatar... Haacked May 20, 2007 4:55 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@Mike - I tried to go through your steps, but w3wpe.exe never gets started. Hmm....
Requesting Gravatar... nitro2k01 May 20, 2007 5:46 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
When hearing about your problems I could just tell you to try open source alternatives. (Which is generally not a bad idea)
But from a pure malfunctioning point of view, FOSS software can laugh you in the face as well.
For example, lately I've been struggling to get a FAMP server (FreeBSD, Apache, MySQL, PHP) to work on a SunBlade machine, a real exotic flavor. (These computers are non-Intel ones that usually run Solaris)
It's a real b****, and so far I haven't got it to work properly. (The server just stops responding whenever I use PHP, with MySQL enabled, even if MySQL isn't used, sometimes it happens under other circumstances too)

Where was I going with that? Oh yeah: Even if you don't choose to convert to Apache, etc., my advice is to let go of Vista for now. My own experience, sadly, is that Windows OS's don't get really mature until at least a year after the release, when initial bugs and holes are fixed.
Requesting Gravatar... Haacked May 20, 2007 6:20 PM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@nitro - I understand the sentiment. But in this case, I have pretty much exactly the same setup at work. Vista running IIS7. So the fact that it doesn't work at home doesn't seem to be a Vista issue (nor an IIS7 issue) per se.

It seems like somehow, my installation at home got corrupted. I could just reinstall IIS fully (someone told me I was missing a step in my previous attempt) but I'm working with a couple people on the IIS 7 team to see if we can't figure out *why* this is happening. Just in case it really is a problem in IIS 7 that they haven't seen before. I'm happy to help out.

As for switching to Apache, not really feasible right this second. I'm trying to get my machine setup at home so I can work from home or work. So I need to be able to run the same webservers we use at work.

However, I believe our product will work on Apache in theory. So it's definitely something I will setup in the future.
Requesting Gravatar... Haacked May 21, 2007 1:58 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@engtech - I was joking about the "helpful" part. I love your blog.
Requesting Gravatar... Brian Chiasson May 21, 2007 7:58 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
HOST table anyone - are you sure that localhost has not been remapped? Here are the settings for localhost in my hosts file (left out the comments).

127.0.0.1 localhost
::1 localhost

I would also check any firewalls to make sure they are not blocking anything.
Requesting Gravatar... Mike Volodarsky May 21, 2007 8:29 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Phil,

The steps may still be applicable assuming the problem is with IIS.

I would try the following:

1) Look in %systemdrive%\WINDOWS\system32\LogFiles\HTTPERR httperrX.log file to see if http.sys is rejecting the requests for some reason

See if requests are causing entries to be written here, and if so, post them.

If not:

2) Follow the steps in the startup-debugging post.

The basic thing we need to confirm is whether the request is getting into the worker process (even if its crashing and going away) - depending on this, the subsequent diagnostics will be much different.

Thanks,

Mike
Requesting Gravatar... Haacked May 21, 2007 8:36 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Mike, I see a bunch of errors like this:

2007-05-20 00:43:20 ::1%0 62458 ::1%0 80 HTTP/1.1 GET / 503 - N/A -

I tried the debugging steps but it never broke into the debugger. It seems like w3wp.exe never starts.
Requesting Gravatar... Haacked May 21, 2007 8:39 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
@Brian, I have the same thing in my hosts file. I tried turning off the firewall during this whole episode.
Requesting Gravatar... you've been HAACKED May 21, 2007 8:38 PM
# The IIS 7 Team Rocks!
The IIS 7 Team Rocks!
Requesting Gravatar... Community Blogs May 21, 2007 9:28 PM
# The IIS 7 Team Rocks!
I recently wrote about some 503 Service Unavailable Errors with IIS 7 that had me completely stumped
Requesting Gravatar... Tom Jul 27, 2007 8:23 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
"Cannot Read Configuration FILE"

That's what the application log shows for me. I haven't gotten a simple page to show on Vista with IIS7 yet either. Definately frustrating.

Why can't it read the config file? Seems like a security setting someplace.

Any Ideas?
Requesting Gravatar... AliM Dec 03, 2007 6:23 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
I was frustrated by the same problem and the following solution worked for me...

Can you check if you have leftover directory for pe-release version of framework in %windir%\Microsoft.Net\Framework (anything like v2.0.* other than v2.0.50727) - if so, delete those directories and restart the app-pool.


Thanks
Requesting Gravatar... Kamil Czajko Jan 18, 2008 12:53 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Hi Guys um I'm sure you would have already taken a look at this but I didn't see it mentioned, do you get any error messages or log entries when restarting the world wide web publishing service and just check that its logging on with the right credentials?

Also bare in mind that some software can actually bind port 80 as well (ie anything that might let you tunnel such as putty or ems, certain trend products, and vmware.)

You may also be blocked in vista if the network you are connected to is not a trusted network (network and sharing center-> customise next to every network it shows, yes EVERY network even ones that are not connected to anything such as VMware virtual adapters.) I like to set it to work zone as this seems to be the most logical but vista will block ports if even one of the networks it displays is set to unknown or public. I had a virtual server running vista and I had to setup a kvm as it would knock me off rdp everytime a virtual machine would be rebooted or it detected any change on the adapter. >.<

Network Discovery may also need to be on.

All the best
Kamil
Requesting Gravatar... Hondo May 23, 2008 10:47 AM
# re: Service Unavailable Errors in IIS 7 Are Killing Me
Run these two commands to see if you have any URL reservations:

Netsh http show servicestate
and
Netsh http show urlacl

If so, you may have to run something *like*:

netsh http delete urlacl url=http://+:80/

What do you have to say?

(will show your gravatar)
Please add 3 and 6 and type the answer here: