Execution order of modules in IIS7

Each request received by IIS 7.0 goes through multiple stages in the IIS request pipeline (read more about request pipeline here). In IIS, request processing move from one stage to the next stage in a fixed sequence. If any of the modules in system.webServer/modules section have subscribed to the event for the current stage then IIS calls each of those modules one by one before moving on to next stage. If there are multiple modules which subscribe to the same event (say RQ_BEGIN_REQUEST), module with higher priority is called first. Native modules can set execution priority for itself in RegisterModule using SetPriorityForRequestNotification. The following code snippet illustrates this for RQ_BEGIN_REQUEST.

HRESULT
RegisterModule(
    DWORD dwServerVersion,
    IHttpModuleRegistrationInfo* pModuleInfo,
    IHttpServer* pGlobalInfo
)
{

    HRESULT hr = S_OK;

    IHttpModuleFactory* pModuleFactory = NULL;
 
    // create module factory

    hr = pModuleInfo->SetRequestNotifications(pModuleFactory, RQ_BEGIN_REQUEST, 0 );
    // some stuff

    hr = pModuleInfo->SetPriorityForRequestNotification(
               
RQ_BEGIN_REQUEST,
                PRIORITY_ALIAS_FIRST);

    // more stuff
    return hr;
}

If multiple modules subscribe to the same event and set the same priority, the module that is listed first in the system.webServer/modules section gets executed first. RQ_EXECUTE_REQUEST_HANDLER is different in this respect as it picks up the ordering from system.webServer/handlers section. Taking the example of StaticFile handler entry from default IIS configuration which looks like the following

<add name="StaticFile" path="*" verb="*" modules="StaticFileModule,DefaultDocumentModule,DirectoryListingModule" resourceType="Either" requireAccess="Read" />

IIS picks one handler for a request but each entry can have multiple modules executed which can appear as comma separated list as is the case for StaticFile handler. StaticFileModule, DefaultDocumentModule and DirectoryListingModule, all three subscribe to RQ_EXECUTE_HANDLER and use default priority. The default ordering of these modules is important to ensure they work correctly. Changing it can have undesirable effects. For example, putting DefaultDocumentModule before DirectoryListingModule will make DirectoryListingModule run first and if directoryBrowse is disabled, it will generate a 403.14 even before DefaultDocumentModule got a chance to check defaultDocument section and pick default file. DirectoryListingModule should execute only if defaultDocument is disabled and putting it after DefaultDocumentModule serve that purpose.

So IIS decides module execution order as following:

  1. Module which subscribe to an earlier pipeline stages execute before the modules which subscribe to a later stage.
  2. Among modules which subscribe for the same event, module with higher priority runs before module with lower priority. RQ_SEND_RESPONSE priority order is opposite of other notifications. Modules with RQ_SEND_RESPONSE priority LAST are run first, followed by priority LOW and so on. You cannot set priority of managed modules. All managed modules assume default priority.
  3. Among modules which subscribe for the same event and with same priority, module appearing first in system.webServer/modules section gets called first (RQ_EXECUTE_REQUEST_HANDLER stage is special and ordering is picked from system.webServer/handlers section and not from system.webServer/modules).

Global modules which subscribe only to global events can use IHttpModuleRegistrationInfo::SetPriorityForGlobalNotification for setting the priority. Step 3 for globalModules is based on ordering in system.webServer/globalModules section. For native modules which subscribe to only request events, ordering in system.webServer/globalModules section is immaterial (even though they appear in globalModules section). IIS just use the globalModules entry to load the module but calls only modules which appear in system.webServer/modules section.

-Kanwal

Changes to compression in IIS7

Compression module provides IIS the capability to serve compressed responses to compression enabled clients. Clients which can accept compressed responses send Accept-Encoding header indicating compression schemes they can handle. If IIS can compress the response using a compression scheme which client can understand, IIS will send a compressed response with Content-Encoding response header indicating the scheme which was used to compress the response.

So request response look something like

REQUEST: **************
GET /static/index.htm HTTP/1.1\r\n
Accept-Encoding: gzip, deflate\r\n
Host: localhost\r\n
Accept: */*\r\n
\r\n
RESPONSE: **************
HTTP/1.1 200 OK\r\n
Content-Type: text/html\r\n
Content-Encoding: gzip\r\n
Last-Modified: Tue, 28 Mar 2006 03:44:06 GMT\r\n
Accept-Ranges: bytes\r\n
ETag: "087e0dc1952c61:0"\r\n
Vary: Accept-Encoding\r\n
Server: Microsoft-IIS/7.0\r\n
X-Powered-By: ASP.NET\r\n
Date: Tue, 13 Jun 2006 02:23:41 GMT\r\n
Content-Length: 438\r\n

This is useful to save network bandwidth and offer clients faster download times. This was first added in IIS6 and could be enabled for static/dynamic content individually. Following set of changes are made to compression modules in IIS7:

  1. Static compression is ON by default in IIS7. Dynamic compression is still OFF by default which can be turned ON for all content by running
    appcmd set config -section:urlCompression /doDynamicCompression:true
  2. In IIS6, static compression was happening on a separate thread. So on receiving a request, first response was uncompressed and IIS used to start a separate thread to compress the file and keep it in compressed files cache. Requests for compressed content reaching IIS after the compression is complete used to receive compressed response. In IIS7, compression happens on the main thread. But to not incur cost of compression for all requests compression happens only for frequently requested content. Definition of frequently requested content is controlled by properties frequentHitThreshold and frequentHitTimePeriod under section system.webServer/serverRuntime. If IIS receives more than threshhold number of requests in frequentlyHitTimePeriod for the same url, IIS will go ahead and compress the file to serve compressed response for the same request which made IIS reach threshhold. This compressed response is saved in the compressed files cache as in IIS6. If compressed response was already present in compression cache, frequentHitThreshhold logic is not applied as compresed content will be picked from cache and there will be no additional cost for compressing the content. Hit count is maintained per URL. So sending first request with Accept-Encoding: gzip and second with deflate will still qualify as frequentlyHit content and IIS will go ahead and compress the response. This will require cachuri.dll to present in the globalModules section as that is the module which keeps URL hit count (More about globalModules someday later).
  3. Temporary compressed files folder has a nested directory structure in IIS7 compared to a flat one in IIS6. We create folder for each apppool in temporary compressed files and then create separate folders for different schemes under each apppool. Under these scheme folders, we create folder structure similar to folder from where the content was picked. So if iisstart.htm from D:\inetpub\wwwroot got compressed using gzip, cache entry will get created in
    D:\inetpub\temp\IIS Temporary Compressed Files\DefaultAppPool\$^_gzip_D^\INETPUB\WWWROOT folder. We ACL the apppool folder with worker process identity to protect the content from worker processes serving some other apppools. Directory is still configurable from config but the default is moved from %windir%\iis temporary compressed files to %SystemDrive%\inetpub\temp\iis temporary compressed files. Also with this change, maxDiskSpaceUsage limit is applied per apppool. So if you had a value of 100MB for HcMaxDiskSpaceUsage in IIS6, that limit was applied to all the compressed content in the compressed files cache. Now this limit applies to compressed files per apppool. If you have 10 apppools and have maxDiskSpaceUsage set to 100MB, total space allocated to compressed files cache is actually 1GB.
  4. As static compression is enabled by default and compression is happening on the main thread, we added on the fly compression shutoff/resume depending on CPU load. Four properties are added to system.webServer/httpCompression section to control this behavior. These are
    – staticCompressionDisableCpuUsage – compression is disabled when average CPU usage over a time period is above this number.
    – staticCompressionEnableCpuUsage – compression is enabled back if average CPU usage over a time period falls below this number.
    – dynamicCompressionDisableCpuUsage and dynamicCompressionEnableCpuUsage enables or disables dynamic compression depending on load of CPU.
    We calculate average CPU utilization every 30 sec.
  5. In IIS7, you can enable/disable compression depending on the content-type of response. In IIS6 this was possible on extension basis which is no longer possible. Now you can have just one entry in the config to enable static or dynamic compression for text/html responses. You no longer need to pick up all extensions which return text/html responses. While configuring these mimeTypes under httpCompression section, you can use * as wildcard. If response type is text/html, we look for an entry for text/html and if found use the corresponding enabled value. If text/html is not found, we look for text/* or */html. If both are present we pick the one which comes first and use that enabled property value. If these are also not found, we look for */* and use the corresponding enabled value. For enabling compression for all content types, add an entry under httpCompression section in applicationHost.config which says
        <staticTypes>
            <add mimeType="*/*" enabled="true" />
        </staticTypes>
  6. Bunch of IIS6 compression properties are removed in IIS7. These are
    – HcMaxQueueLength. As compression is happening on main thread, queue length property is not used anymore and is removed.
    – HcCreateFlags property which was used to pass a value to gzip.dll (indicating gzip or deflate compression) is removed as it was not useful for the customers.
    – HcFileExtensions and HcScriptFileExtensions are removed as we no longer enable/disable compression based on extension.
    – HcIoBufferSize and HcCompressionBufferSize properties are removed as we believe those are not useful for customers.
    – HcDoOnDemandCompression is removed. For enabling static compression, only doStaticCompression property is required to be set.
    – HcFilesDeletedPerDiskFree is no longer there in IIS7. We just delete as many files as required to get to 90% of the maxDiskSpaceUsage. We pick least recently used files to delete.
    – HcPriority is removed as scheme to use when multiple ones appear in Accept-Encoding header of request is picked depending on scheme which is appear first in Accept-Encoding header (assuming no q factor). This is as per HTTP specs.

     

    I think ability to add new schemes in config is not very useful. I have not seen any compression scheme implementation outside Microsoft which can be used with IIS. If you want to implement your own scheme, you need to write a dll with following exports
    – Compress
    – CreateCompression
    – DeInitCompression
    – DestroyCompression
    – InitCompression
    – ResetCompression
    If you want signatures of these functions, write to me and i can take your case to the management. Most probably you will get the signatures.

  7. maxDiskSpaceUsage entry is changed to be configured in MB rather than bytes. We realized that people don't really want to configure the limit to the byte level but the main reason was we have the limit as UINT and we didn't want customers to set it to a value which cannot be stored in UINT. With large disks today, having a large value won't be uncommon.
  8. With static compression enabled by default, now we only cache compressed responses in the kernel (HTTP.Sys). So if compression is enabled for a file but the current request didn't contain Accept-Encoding header (or compression didn't happen because it was the first request), we don't tell http.sys to cache it. Only compressed response is cached in the kernel for which compression is enabled. Dynamically compressed responses are not cached in any of the caches (even in compressed files as was in IIS6).
  9. Deflate is removed in default configuration but the functionality is still present in gzip.dll. To add deflate scheme, add the following in httpCompression section:
        <scheme name="deflate" dll="%Windir%\system32\inetsrv\gzip.dll" />
    You can use following appcmd command.
    appcmd set config /section:httpCompression /+[name='deflate',dll='%Windir%\system32\inetsrv\gzip.dll']
  10. As static compression is enabled by default, default value of staticCompressionLevel is changed from 10 to 7.

Most of the compression properties (present under system.webServer/httpCompression) can be set in applicationHost.config for the server and cannot be configured on URL basis. We might have per apppool properties someday and if we end up doing per apppool properties, properties under httpCompression will be configurable per apppool. doStaticCompression and doDynamicCompression boolean properties under system.webServer/urlCompression section can be set at URL level which can be used to enable/disable compression per URL. This is similar to what was in IIS6.

I think i have mentioned all the changes that are coming in IIS7 as far as compression is concerned. If i realize that i missed something, i will do another post.

-Kanwal