Migrating FastCGI configuration from IIS 5.1/6.0 to IIS 7.0/7.5

Problem

As you know FastCGI functionality on IIS 5.1 and IIS 6.0 is provided by FastCGI ISAPI extension which is available as an independent download. On IIS 7.0 and beyond, FastCGI functionality is provided by IIS FastCGI module which comes with the operating system. These components use different configuration stores to store the settings that affect their behavior. FastCGI ISAPI uses an INI file named fcgiext.ini as its configuration store whereas FastCGI module keeps configuration in applicationHost.config in xml format with rest of IIS configuration. Before, migrating from IIS6 to IIS7 involved migrating fcgiext.ini configuration to FastCGI module manually as there was no automated way to do that. MSDeploy only had functionality to migrate metabase configuration but because FastCGI ISAPI configuration is kept separately in an INI file, that wasn’t possible.

Solution

MSDeploy v1 RTW which is released today include a new provider named fcgiextConfig to automate task of migrating FastCGI ISAPI INI configuration to FastCGI module section. FcgiextConfig provider accepts a path which can be “APPHOST” or ”INI”. When path “APPHOST” is specified, configuration from system.webServer/fastCgi section in applicationHost.config is read. Path “APPHOST” should only be used on systems running FastCGI module (i.e. IIS7 and beyond). When path “INI” is specified, this provider reads the configuration from file “%windir%\system32\inetsrv\fcgiext.ini” and produce xml which looks like FastCGI module configuration. Once INI settings are mapped to produce the xml, msdeploy engine can compare configuration of FastCGI ISAPI with FastCGI module and make changes to FastCGI module configuration as required. Again, path “INI” can only be used on a machine which is running FastCGI ISAPI.

Examples

Dump command to dump INI settings looks like following.
        msdeploy –verb:dump –source:fcgiExtConfig=ini –xml
        msdeploy –verb:dump –source:fcgiExtConfig=apphost –xml

Sync command to move INI settings to FastCGI module section is following.
        msdeploy –verb:sync –source:fcgiExtConfig=ini –dest:fcgiExtConfig=apphost -whatif

Details

  Here is how INI settings are mapped to system.webServer/fastCgi settings. For each entry   “<extension>:<optionalsiteid>=<sectionname>” under [Types] section of fcgiext.ini, we read the settings under section [<sectionname>]. If optional site id is present, we look for an entry for the same extension but without site id. If an entry is found, we use configuration under this entry as base set of settings for the site specific entry. So if “php=basephp” and “php:1=site1” entries are present in INI, settings under section [basephp] provide base values and then settings under section [site1] override base settings to generate configuration for “php:1” process pool. If a property is not present in the INI file, we assume the default value as assumed by FastCGI ISAPI extension. A FastCGI process pool entry is created for each INI entry with a valid ExePath. Various INI file settings are mapped to a FastCGI process pool as below.
  

fcgiext.ini configuration setting system.webServer/fastCgi/application property
ExePath fullPath
Arguments arguments
QueueLength queueLength
MaxInstances maxInstances
IdleTimeout idleTimeout
ActivityTimeout activityTimeout
RequestTimeout requestTimeout
InstanceMaxRequests instanceMaxRequests
FlushNamedPipe flushNamedPipe
Protocol protocol
RapidFailsPerMinute rapidFailsPerMinute
EnvironmentVars environmentVariables
ResponseBufferLimit Ignored
IgnoreExistingFiles Ignored
IgnoreDirectories Ignored
UnhealthyOnQueueFull Ignored


If the source version is FastCGI ISAPI 1.5, some additional properties get picked and mapped as below.
 

StderrMode stderrMode
MonitorChangesTo MonitorChangesTo
SignalBeforeTerminateSeconds signalBeforeTerminateSeconds


If FastCGI ISAPI version on source and FastCGI module version on destination are not compatible, fcgiextConfig provider will block the migration. Note that fcgiextConfig provider doesn’t take care of migrating the relevant IIS ScriptMaps. You can migrate ScriptMaps using a separate msdeploy migrate operation.

Hope this helps.
Kanwal

Getting both FastCGI module and AppPool CPULimit to work

FastCGI module available in IIS 7.0 did not work when CPULimit for the application pool was enabled. This was because when CPULimit feature was enabled WAS uses job objects to track CPU usage of worker processes. WAS keeps a job object for each application pool and all worker processes which belong to that application pool are made part of that job object. FastCGI module uses job objects to make sure that there are no orphan child processes left when worker process goes away. Note that FastCGI requires a per process job object while WAS requires a per application pool job object. For this reason, it was not possible to make WAS take care of killing FastCGI child processes when worker process dies. Because a process can only be part of one job object on windows and also the fact that a child process becomes part of the job object if parent process is already part of existing job object, FastCGI module was unable to make child processes part of another job object when CPULimit was turned on and used to fail with error “Unable to place a FastCGI process in a JobObject. Try disabling the Application Pool CPU Limit feature”.

Customers who used CPULimit feature complained about limitation. We recommended customers to use WSRM (windows system resource manager) but some wanted to continue using CPULimit functionality. Due to requests from these customers we have fixed this issue in Windows 7 (Win2K8 R2) and also released a QFE for Win2K8 release. In the fix, WAS lets FastCGI make its child processes part of other job object. Also FastCGI module keeps track of CPU usage of its child processes and periodically report CPU usage by children back to WAS which then adds that to the application pool CPU usage count.

You can read more about this here and here. QFE can be downloaded from here.

Hope this helps.

Kanwal

FastCGI ISAPI 1.5 Beta for WinXP and Win2K3

IIS team today released FastCGI ISAPI 1.5 Beta for WinXP and Win2K3 which has some very nice additions to existing FastCGI ISAPI functionality. Following additions have been made to FastCGI ISAPI 1.0.

1. Few features we added to FastCGI module in IIS 7.5 have been added to FastCGI ISAPI 1.5 as well. These include MonitorChangesTo, StderrMode and Real-time tuning. Read more about these here.

2.
Few customers complained about IIS terminating the FastCGI processes abruptly (on running into IdleTimeout, InstanceMax etc) without giving them a chance to run cleanup code in the FastCGI application. In FastCGI ISAPI 1.5, we have added ability to get a signal from IIS whenever we are about to terminate a child process. To enable this, you need to set SignalBeforeTerminateSeconds property in fcgiext.ini to greater than 0. When this functionality is enabled, IIS will create an inheritable event and pass its handle value to child process as value of _FCGI_SHUTDOWN_EVENT_ environment variable. If FastCGI ISAPI ever encounters a situation (like worker process shutting down, fcgiext.ini changed, file being monitored changed etc) and is required to terminate the child process, it will first signal this event and wait for a maximum of SignalBeforeTerminateSeconds for process to terminate on its own. On detecting event being signaled, child processes can terminate themselves cleanly. If child process are still alive after the wait period, IIS will forcibly terminate them.

3.
Name of the named pipe through which communication with FastCGI process is taking place is communicated as value of _FCGI_X_PIPE_ environment variable.

4. Many customers faced trouble with FastCGI 1.0 because they accidently added some invalid configuration to fcgiext.ini and couldn’t decipher the cryptic error message they got from FastCGI ISAPI. In this release, if we see an invalid property present or invalid value of an enum type property, we exactly tell you what’s wrong in fcgiext.ini and where the error is.

5. FastCGI 1.0 had very strict checking on validity of response headers. We got bunch of reports from customers who complained about this difference of behavior compared to other web servers. In this release, we have removed these checks. If we find an invalid response header, we silently drop it from the response. This behavior matches behavior of other web servers.

6. Customers easily ran into ActivityTimeout especially while running install.php code of popular PHP applications. So we have changed default value of ActivityTimeout from 30 seconds to 70 seconds.

Below are the download links to FastCGI ISAPI 1.5 Beta. If you are using WebPI, FastCGI 1.5 is available under “what’s new” section. You can upgrade from FastCGI ISAPI 1.0 to FastCGI ISAPI 1.5 Beta or do a fresh install today. If you any feedback or trouble using FastCGI ISAPI 1.5, please report it on forums.

X86 – http://download.microsoft.com/download/C/7/8/C783108B-2FA7-4245-835C-E0B887A394D1/fcgisetup32.msi
 

X64 – http://download.microsoft.com/download/5/9/6/596CF5CB-9581-48A5-9205-4D9E3B4FB9BD/fcgisetup64.msi

 

We also worked with Mike from Coast Research who fixed many issues in libfcgi.dll. Also he made changes to libfcgi.dll to make it handle SIGTERM properly using SignalBeforeTerminate functionality we added to FastCGI. Mike kindly released updated libfcgi (called libfcgi2.dll) so that others can now write a FastCGI application with much less pain. You can download libfcgi2.dll from here. Mike has published samples and documentation on how to use libfcgi2.dll on www.coastrd.com. Complete white paper on additions to libfcgi2.dll can be found here.

Thanks,

Kanwal

Using advanced logging to log custom module data

Advanced logging module which media team released few days ago uses IIS tracing subsystem and allow module developers to log custom data in W3C compatible format using familiar IHttpTraceContext interface. If you are a module developer and want to generate W3C style logs for requests with custom data, doing it with advanced logging module is very easy. All you need to do is call IHttpTraceContext::RaiseTraceEvent passing data you want to publish. Your module installer can then configure advanced logging module to make it dump this data in a log file and you have complete logging solution without ever have to deal with log files yourself. Let’s see what it takes to make use of this functionality.

Generating trace information in your module

I have a simple class below which hides some of the complexity of IIS tracing subsystem. You can use most of the code below directly in your module. Just change area GUID and also code under RaiseEvent to dump your own custom data.

#include <httpserv.h>
#include <httptrace.h>

//
// Start of the new provider class AdvancedLoggingTraceProvider,
//
// GUID: {f41fdbf7-1b0a-45e9-9666-2918b0a9d144}
// NOTE:
// You must use this provider GUID for advanced logging to
// process events generated by your module
//
// Description: IIS: Advanced Logging
//
class AdvancedLoggingTraceProvider
{
public:
    static
    LPCGUID
    GetProviderGuid(
        VOID
    )
    {
        static const GUID AdvLoggingProviderGuid =
          {0xf41fdbf7,0x1b0a,0x45e9,{0x96,0x66,0x29,0x18,0xb0,0xa9,0xd1,0x44}};
        return &AdvLoggingProviderGuid;
    };

    enum enumAreaFlags
    {
        IISAdvancedLoggingGeneral = 0x0000
    };

    static
    LPCWSTR
    TranslateEnumAreaFlagsToString(
        enum enumAreaFlags EnumValue
    )
    {
        switch( (DWORD) EnumValue )
        {
            case 0x0000: return L"IISAdvancedLoggingGeneral";
        }
        return NULL;
    };

    static
    BOOL
    CheckTracingEnabled(
        IHttpTraceContext * pHttpTraceContext,
        enumAreaFlags       AreaFlags,
        DWORD               dwVerbosity
    )
    {
        HRESULT                  hr;
        HTTP_TRACE_CONFIGURATION TraceConfig;
        TraceConfig.pProviderGuid = GetProviderGuid();
        hr = pHttpTraceContext->GetTraceConfiguration( &TraceConfig );
        if ( FAILED( hr )  || !TraceConfig.fProviderEnabled )
        {
            return FALSE;
        }
        if ( TraceConfig.dwVerbosity >= dwVerbosity &&
             (  TraceConfig.dwAreas == (DWORD) AreaFlags || 
             ( TraceConfig.dwAreas & (DWORD)AreaFlags ) == (DWORD)AreaFlags ) )
        {
            return TRUE;
        }
        return FALSE;
    };
};

//
// Start of the new event class IISAdvancedLoggingGeneralEvents
// GUID: {ffa89186-ec59-42c1-988f-bbbd0ee48d4f}
// Use any random area GUID
// Description: Advanced Logging general events
//
class IISAdvancedLoggingGeneralEvents
{
public:
    static LPCGUID GetAreaGuid( VOID )
    {
        static const GUID AreaGuid =
          {0xffa89186,0xec59,0x42c1,{0x98,0x8f,0xbb,0xbd,0x0e,0xe4,0x8d,0x4f}};
        return &AreaGuid;
    };

    class SAMPLE_TRACE_EVENT
    {
    public:
        static
        HRESULT
        RaiseEvent(
            IHttpTraceContext * pHttpTraceContext,
            LPCGUID      pContextId,
            LPCWSTR      pSample_Data1,
            LPCWSTR      pSample_Data2
        )
        {
            HTTP_TRACE_EVENT Event;
            Event.pProviderGuid  = AdvancedLoggingTraceProvider::GetProviderGuid();
            Event.dwArea         =  AdvancedLoggingTraceProvider::IISAdvancedLoggingGeneral;
            Event.pAreaGuid      = IISAdvancedLoggingGeneralEvents::GetAreaGuid();
            Event.dwEvent        = 1;
            Event.pszEventName   = L"SAMPLE_TRACE_EVENT";
            Event.dwEventVersion = 1;
            Event.dwVerbosity    = 0;
            Event.cEventItems    = 6;
            Event.pActivityGuid  = NULL;
            Event.pRelatedActivityGuid = NULL;
            Event.dwTimeStamp    = 0;
            Event.dwFlags        = HTTP_TRACE_EVENT_FLAG_STATIC_DESCRIPTIVE_FIELDS;

            // pActivityGuid, pRelatedActivityGuid, Timestamp to be filled in by IIS

            HTTP_TRACE_EVENT_ITEM Items[ 3 ];
            Items[ 0 ].pszName = L"ContextId";
            Items[ 0 ].dwDataType = HTTP_TRACE_TYPE_LPCGUID; // mof type (object)
            Items[ 0 ].pbData = (PBYTE) pContextId;
            Items[ 0 ].cbData = 16;
            Items[ 0 ].pszDataDescription = NULL;
            Items[ 1 ].pszName = L"Sample_Data1";
            Items[ 1 ].dwDataType = HTTP_TRACE_TYPE_LPCWSTR; // mof type (string)
            Items[ 1 ].pbData = (PBYTE) pSample_Data1;
            Items[ 1 ].cbData  =
                 ( Items[ 1 ].pbData == NULL )? 0 : ( sizeof(WCHAR) * (1 + (DWORD) wcslen( (PWSTR) Items[ 1 ].pbData  ) ) );
            Items[ 1 ].pszDataDescription = NULL;
            Items[ 2 ].pszName = L"Sample_Data2";
            Items[ 2 ].dwDataType = HTTP_TRACE_TYPE_LPCWSTR; // mof type (string)
            Items[ 2 ].pbData = (PBYTE) pSample_Data2;
            Items[ 2 ].cbData  =
                 ( Items[ 2 ].pbData == NULL )? 0 : ( sizeof(WCHAR) * (1 + (DWORD) wcslen( (PWSTR) Items[ 2 ].pbData  ) ) );
            Items[ 2 ].pszDataDescription = NULL;

            Event.pEventItems = Items;
            pHttpTraceContext->RaiseTraceEvent( &Event );

            return S_OK;
        };

        static
        BOOL
        IsEnabled(
            IHttpTraceContext *  pHttpTraceContext
        )
        {
            return AdvancedLoggingTraceProvider::CheckTracingEnabled(
                                 pHttpTraceContext,
                                 AdvancedLoggingTraceProvider::IISAdvancedLoggingGeneral,
                                 0 );
        }
    };
};

Once you have class(es) similar to SAMPLE_TRACE_EVENT defined for the event(s) you want to generate, you can just call RaiseEvent in your module. Before generating the event you should call IsEnabled() to check if a relevant trace provider is present with tracing enabled. Here is what you will write in your module.

IHttpTraceContext* pTraceContext = pContext->GetTraceContext();

if( IISAdvancedLoggingGeneralEvents::SAMPLE_TRACE_EVENT::IsEnabled( pTraceContext ) == TRUE )
{
    IISAdvancedLoggingGeneralEvents::SAMPLE_TRACE_EVENT::RaiseEvent( pTraceContext,
                                                                     NULL,
                                                                     “Data1”,
                                                                     “Data2” );
}

Configuring advanced logging to generate logs

 


Now to dump data generated by your module in advanced logging logs, you need to create log definition and specify event names as field names in advanced logging configuration. To do this, follow the steps as below in advanced logging UI.

1.       Go to advanced logging UI and create a new log definition.

2.       Specify details and then add additional fields which your module is generating. In the sample above these are Sample_Data1 and Sample_Data2. As advanced logging require a unique name to identify the event uniquely, we recommend people to use “<ModuleName>_” as prefix in the event names.

3.       Restart w3svc (this requirement will be removed in the next release of advanced logging).

That’s it. Log file produced will have single entry for each request received by the server. If a request gets short circuited and never reaches your module code where you generate the trace event, Sample_Data1 and Sample_Data2 field values will be blank strings. To log only requests which reached your module’s RaiseEvent call, make Sample_Data1 and Sample_Data2 fields “required” in advanced logging configuration. Note that ability to mark a field “required” is not there in the UI yet. You can edit the IIS configuration file manually or you other tools like appcmd, configuration editor to do it. More detailed steps to configure advanced logging are available here.

Hope this helps.
Kanwal

Implementing IAppHostPathMapper in C

Few days ago I was required to implement IAppHostPathMapper interface in native C to map configuration path MACHINE/WEBROOT/APPHOST to DefaultAppPool.config and struggled with finding good documentation. With help of some incomplete, hard to find documentation and some head banging here is what worked for me. Hopefully this will be useful for few others J.

#include <ahadmin.h>

//
// PathMapper object structure with VTable pointer and reference counter
// Add other private data in this structure
//
typedef struct TestPathMapper
{
    IAppHostPathMapperVtbl * lpVtbl;
    ULONG                    cRef;
} TestPathMapper;

//
// Method definitions
//
STDMETHODIMP TestPathMapper_QueryInterface(TestPathMapper *, REFIID, LPVOID FAR *);
STDMETHODIMP_(ULONG) TestPathMapper_AddRef(TestPathMapper *);
STDMETHODIMP_(ULONG) TestPathMapper_Release(TestPathMapper *);
STDMETHODIMP TestPathMapper_MapPath(TestPathMapper *, BSTR, BSTR, BSTR *);

//
// IAppHostPathMapper VTable structure
//
static const IAppHostPathMapperVtbl vtblTestPathMapper =
{
    TestPathMapper_QueryInterface,
    TestPathMapper_AddRef,
    TestPathMapper_Release,
    TestPathMapper_MapPath
};

STDMETHODIMP
TestPathMapper_QueryInterface(
    TestPathMapper * pThis,
    REFIID           riid,
    LPVOID FAR *     lppvObj
)
{
    if ( !pThis || pThis->lpVtbl != &vtblTestPathMapper || !lppvObj )
    {
        return E_INVALIDARG;
    }

    if ( !memcmp(riid, &IID_IUnknown, sizeof( IID ) ) ||
         !memcmp(riid, &IID_IAppHostPathMapper, sizeof( IID ) ) )
    {
        pThis->lpVtbl->AddRef( pThis );
        *lppvObj = pThis;

        return S_OK;
    }

    *lppvObj = NULL;
    return E_NOINTERFACE;
}

STDMETHODIMP_(ULONG)
TestPathMapper_AddRef(
    TestPathMapper * pThis
)
{
    if (!pThis || pThis->lpVtbl != &vtblTestPathMapper)
    {
        return 1;
    }

    return InterlockedIncrement( &pThis->cRef );
}

STDMETHODIMP_(ULONG)
TestPathMapper_Release(
    TestPathMapper * pThis
)
{
    LONG cRef;   
    if ( !pThis || pThis->lpVtbl != &vtblTestPathMapper )
    {
        return 1;
    }

    cRef = InterlockedDecrement( &pThis->cRef );
    if (cRef == 0)
    {
        pThis->lpVtbl->Release(pThis);
        pThis->lpVtbl = NULL;
        free( pThis );
    }

    return cRef;
}

STDMETHODIMP
TestPathMapper_MapPath(
    TestPathMapper * pThis,
    BSTR             bstrConfigPath,
    BSTR             bstrMappedPhysicalPath,
    BSTR *           pbstrNewPhysicalPath
)
{
    BSTR bstrNewPath  = NULL;
    if ( !pThis || pThis->lpVtbl != &vtblTestPathMapper )
    {
        return E_INVALIDARG;
    }

    if( wcscmp( bstrConfigPath, L"MACHINE/WEBROOT/APPHOST" ) == 0 )
    {
        bstrNewPath = SysAllocString( L"%systemdrive%\\inetpub\\temp\\apppools\\DefaultAppPool.config" );
    }
    else
    {
        bstrNewPath = SysAllocString( bstrMappedPhysicalPath );
    }

    if( bstrNewPath == NULL )
    {
        return E_OUTOFMEMORY;
    }

    *pbstrNewPhysicalPath = bstrNewPath;
    return S_OK;
}

int _tmain(
    int argc,
    _TCHAR* argv[]
)
{
    HRESULT   hr        = S_OK;
    DWORD     dwCount   = 0;
    VARIANT   varUnknown;

    TestPathMapper *            pTestPathMapper    = NULL;
    IAppHostAdminManager *      pAMgr              = NULL;
    IAppHostElement *           pElement           = NULL;
    IAppHostElementCollection * pElementCollection = NULL;   

    BSTR bstrPathMapper       = SysAllocString( L"pathMapper" );
    BSTR bstrConfigPath       = SysAllocString( L"MACHINE/WEBROOT/APPHOST" );
    BSTR bstrSitesSectionName = SysAllocString( L"system.webServer/caching" );

    if( bstrPathMapper == NULL || bstrConfigPath == NULL || bstrSitesSectionName == NULL )
    {
        hr = E_OUTOFMEMORY;
        goto Finished;
    }

    hr = CoInitializeEx(NULL, COINIT_MULTITHREADED);
    if( FAILED(hr) )
    {
        goto Finished;
    }

    hr = CoCreateInstance( &CLSID_AppHostAdminManager,
                           NULL,
                           CLSCTX_INPROC_SERVER,
                           &IID_IAppHostAdminManager,
                           (void**) &pAMgr );
    if( FAILED( hr ) )
    {
        goto Finished;
    }

    //
    // Create pathMapper structure
    //
    pTestPathMapper = (TestPathMapper *)malloc( sizeof( TestPathMapper ) );
    if( pTestPathMapper == NULL )
    {
        hr = E_OUTOFMEMORY;
        goto Finished;
    }

    pTestPathMapper->lpVtbl = &vtblTestPathMapper;
    pTestPathMapper->cRef   = 1;

    VariantInit(&varUnknown);

    V_VT(&varUnknown)      = VT_UNKNOWN;
    V_UNKNOWN(&varUnknown) = pTestPathMapper;

    //
    // Set pathMapper metadata
    //
    hr = pAMgr->lpVtbl->SetMetadata( pAMgr, bstrPathMapper, varUnknown );
    if( FAILED( hr ) )
    {
        goto Finished;
    }

    //
    // Get the sites section and get number of sites
    //
    hr = pAMgr->lpVtbl->GetAdminSection( pAMgr, bstrSitesSectionName, bstrConfigPath, &pElement );
    if( FAILED( hr ) || pElement == NULL )
    {
        goto Finished;
    }

    hr = pElement->lpVtbl->get_Collection( pElement, &pElementCollection );
    if( FAILED( hr ) || pElementCollection == NULL )
    {
        goto Finished;
    }

    hr = pElementCollection->lpVtbl->get_Count( pElementCollection, &dwCount );
    if( FAILED( hr ) )
    {
        goto Finished;
    }

    wprintf( L"%d\n", dwCount );

Finished:

    VariantClear(&varUnknown);

    if( pElementCollection != NULL )
    {
        pElementCollection->lpVtbl->Release( pElementCollection );
        pElementCollection = NULL;
    }

    if( pElement != NULL )
    {
        pElement->lpVtbl->Release( pElement );
        pElement = NULL;
    }

    if( pAMgr != NULL )
    {
        pAMgr->lpVtbl->Release( pAMgr );
        pAMgr = NULL;
    }

    SysFreeString( bstrPathMapper );
    SysFreeString( bstrConfigPath );
    SysFreeString( bstrSitesSectionName );

    CoUninitialize();
    return 0;
}

Thanks.
Kanwal