Wednesday, June 27, 2012

Performing Control Flow task in SSIS Package

Steps to create Control Flow in SSIS package


Below is an example design for control flow
Get a SQL task and name it as 'SQL Insert Audit' to insert audit values and update the script in 'SQL Statement' field as given below:
exec spu_insert_audit ?,?, ?,?,?,? OUT

and go to the Parameter Mapping option and add the variables as shown below:
next, get a SQL task and name it as 'SQL Drop All FK' and update the script in 'SQL Statement' field as given below:
exec [dbo].[sp_ExecDropAndRebuildSql]  '','','drop','r' 

now get an 'Execute Package Task' and name it as 'EPT package_name'. In that, go to Package option and select the Location field as 'File System'. In Connection field, select the package name. See below:
now go to the variable option and create variables as shown below:
here the value path for the variable name 'sPkgFolder' is the place where the packages saved.

next we can see the package_name.dtsx created in the Connection Managers area. Select the package name and press F4 to go to the property window.
In Property window, go to the Expression field and select the property as 'ConnectionString' and expression as '@[User::sPkgFolder]+ @[User::sPkgFileNameStaging]'.  Drag the variables from the available list and form the script. Click the Evaluate Expression' button to confirm the expression. See below screen:

do the same step to connect all the packages in control flow as showed in the example design of control flow package in top.

Finally need to get a SQL task to rebuild the Foreign Key. Get a SQL task and name it as 'SQL Rebuild All FK' and update the script in 'SQL Statement' field as given below:
exec [dbo].[sp_ExecDropAndRebuildSql]  '','','create','F'

Performing Audit Task Summary in SSIS Package

Step to create Audit Task Summary package in SSIS


Get DFT Task and name it as "DFT_Load DBName_AuditTaskSummary".
Get into the DFT task and Design package as given below:
Now get into the OLE_Source task and write the SQL command text as given below to get the audit data from AuditTask table.

SELECT GETDATE() AS SummaryDate
              ,MAX(AuditTaskId) AS maxAuditTaskId
              ,MAX(AuditId) AS MaxAuditId
              ,PackageName
             ,TableName
 FROM dbo.AuditTask
 GROUP BY PackageName , TableName


Next, go to lookup task and name it as 'LKP AuditTaskSummary and set as "Redirect rows to no match output" to handle rows with no matching entries.
Go to 'Connection' option and write the below query in "Use results of an SQL query:" filed.


SELECT TableName
              ,PackageName
  FROM dbo.AuditTaskSummary


Next go to the 'Columns' option and map the cloumns as shown below:
From Lookup task, connect the Lookup Match Output to 'OLE DB Command' task. In that task, go the 'Component Properties' section and write the below query in the SqlCommand field.

Update dbo.AuditTaskSummary 
 Set SummaryDate = getDate(),
       MaxAuditTaskId = ?
where tableName = ? and PackageName = ?


And go to 'Column Mapping' section and map the columns as shown below. Check the exact data type match before map it.
Next, from the Lookup task, connect the Lookup No Match Output to 'OLE Destination named as 'OLE Dest_AuditTaskSummary' through Row Count task. Select the 'dbo.AuditTaskSummary' table and map the columns accordingly.

Performing Audit Function in Event Handlers in SSIS Package


Steps and scripts to create Audit function in Event Handlers tab in a SSIS package.

Create variables in package for declared values

--In the Event Handlers tab – do the below steps

Select "OnTaskFailed" - get Script Task and name it as 'SRC SetOnTaskFailed'– in the ReadWriteVariables option – select "User::Status" and this User::Status should be created in variable as "Status" and the value as "Completed".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
Dts.Variables["User::Status"].Value = "OnTaskFailed";
Dts.TaskResult = (int)ScriptResults.Success;
}

Select "OnError" - get Script Task and name it as 'SRC SetOnError'– in the ReadWriteVariables option – select "User::Status" and this User::Status should be created in variable as "Status" and the value as "Completed".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
Dts.Variables["User::Status"].Value = "OnError";
Dts.TaskResult = (int)ScriptResults.Success;
}

From the SRC SetOnError – get a line to connect an Exe SQL task and name it as 'SQL Recreate PK_Index_of_Target'.

In connector – select the Evalution Operation as "Constraint"
and Value as "Success".

Also select the Multiple Constraints with "OR" option.

And connect line to – "SQL Recreate PK_Index_of_Target" (Exe SQL Task)

In which, do as below:

--write the below script to insert values in Audit
EXEC [dbo].[sp_ExecDropAndRebuildSql] ?, ?, 'create', 'p,i'

And go the Parameter Mapping option and select as below

variableName      Direction    DataType  ParameterName  ParameterSize
User::SchemaName  Input      Varchar          0                   -1
User::TableName     Input      Varchar          1                   -1

Select "OnPreExecute" – get Script Task - In the ReadOnlyVariables option – select "System::SourceName,Uder::AuditId,System::PackageName" and in the ReadWriteVariables option – select "User::IsPackageLevel,User::IsParentValue" and this User::IsPackageLevel,User::IsParentValue and UserAuditId should be created in variable as "IsPackageLevel" with value as "False","IsParentValue" with value as "False" and "AuditId" with value as "-1".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
if (Dts.Variables["System::SourceName"].Value.Equals(Dts.Variables["System::PackageName"].Value))
{
Dts.Variables["User::IsPackageLevel"].Value = true;
if ((int)Dts.Variables["User::AuditId"].Value == -1)
{
Dts.Variables["User::IsParentValue"].Value = false;
}
else
{
Dts.Variables["User::IsParentValue"].Value = true;
}
}
else
{

Dts.Variables["User::IsPackageLevel"].Value = false;
}

Dts.TaskResult = (int)ScriptResults.Success;
}

From the SRC Intial – get two line to connect two Exe SQL task

"SQL Insert [DBName]_Audit (AND) SQL Insert [DBName]_AuditTask"

In connector I – select the Evalution Operation as "Expression and Constraint"
and Value as "Success" and Expression as "@IsPackageLevel && !@IsParentValue".

Also select the Multiple Constraints with "AND" option.

And connect line to – "SQL Insert Imports_Audit" (Exe SQL Task)

In which, do as below:

--write the below script to insert values in Audit
exec spu_insert_audit ?,?, ?,?,?,? OUT

And go the Parameter Mapping option and select as below

variable Name            Direction   DataType  ParameterName  ParameterSize
User::AuditId              Output       Long             5                      -1
System::StartTime       Input         Date             0                      -1
System::Versionbuild   Input          Long             1                     -1
System::UserName       Input        Varchar          2                      -1
System::PackageName Input         Varchar          3                      -1
System::MachineName Input         Varchar          4                      -1

In connector II – select the Evalution Operation as "Expression and Constraint"
and Value as "Success" and Expression as

"!(SUBSTRING(@[System::SourceName],1,4)=="FELC") && ! (SUBSTRING(@[System::SourceName],1,3)=="SRC") && !@IsPackageLevel".

Also select the Multiple Constraints with "AND" option.

And connect line to – "SQL Update Imports_AuditTask" (Exe SQL Task)

In which, do as below:

--write the below script to insert values in AuditTask
exec spu_insert_audit_task ?,?,?,?,?,?,?,?,? OUT

And go the Parameter Mapping option and select as below

variable Name             Direction   DataType   ParameterName   ParameterSize
User::AuditId                Input         Long              0                     -1
User::AuditTaskId          Output       Long             8                      -1
System::StartTime         Input         Date              1                     -1
System::Versionbuild     Input         Long              2                      -1
System::UserName        Input        Varchar           3                      -1
System::PackageName   Input        Varchar           4                      -1
System::MachineName    Input       Varchar           5                      -1
System::SourceName      Input       Varchar           6                       -1
User::TableName           Input        Varchar          7                       -1

Select "OnPostExecute" – get Script Task - In the ReadOnlyVariable option – select "System::SourceName,System::PackageName" and in the ReadWriteVariables option – select "User::IsPackageLevel" and this User::IsPackageLevel should be created in variable as "IsPackageLevel" and the value as "False".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
if (Dts.Variables["System::SourceName"].Value.Equals(Dts.Variables["System::PackageName"].Value))
{
Dts.Variables["User::IsPackageLevel"].Value = true;
}
else
{
Dts.Variables["User::IsPackageLevel"].Value = false;
}
Dts.TaskResult = (int)ScriptResults.Success;
}

From the SRC IsPackageLevel – get two line to connect two Exe SQL task

"SQL Update [DBName]_Audit (AND) SQL Update [DBName]_AuditTask"

In connector I – select the Evalution Operation as "Expression and Constraint"
and Value as "Success" and Expression as "@IsPackageLevel && !@IsParentValue".

Also select the Multiple Constraints with "AND" option.

And connect line to – "SQL Update Imports_Audit" (Exe SQL Task)

In which, do as below:

--write the below script to update values in Audit
UPDATE [dbo].[Audit]
SET [Status] = 'Completed'
,[EndTime] = GETDATE()
Where AuditId= -1
GO

Go to Expression – select the "SqlStatementSource" option and write below query:

"UPDATE [dbo].[Audit]
SET [Status] = '"+ @[User::Status] +"'
,[EndTime] = GETDATE()
Where AuditId= "+ (DT_WSTR, 20) (@[User::AuditId]) +"
GO
"

In connector II – select the Evalution Operation as "Expression and Constraint"
and Value as "Success" and Expression as "!@IsPackageLevel".

Also select the Multiple Constraints with "AND" option.

And connect line to – "SQL Update Imports_AuditTask" (Exe SQL Task)

In which, do as below:

--write the below script to update values in AuditTask
UPDATE [dbo].[AuditTask]
SET [Status] = 'Completed'
,[EndTime] = GETDATE()
,[RowsAffected]= 0
,[LastModDate]=''
Where AuditTaskId= '0'
GO

Go to Expression – select the "SqlStatementSource" option and write below query:

"UPDATE [dbo].[AuditTask]
SET [Status] = '"+@[User::Status]+"'
,[EndTime] = GETDATE()
,[RowsAffected]= "+ (DT_WSTR, 20)( @[User::RowsAffected]) +"
,[LastModDate]='"+@[User::LastModDate] +"'
Where AuditTaskId= '"+ (DT_WSTR, 20)(@[User::AuditTaskId]) +"'
GO"

==================================================

To Create Audit function in Control flow

--In the Event Handlers tab – do the below steps

Select "OnTaskFailed" - get Script Task – in the ReadWriteVariables option – select "User::Status" and this User::Status should be created in variable as "Status" and the value as "Completed".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
Dts.Variables["User::Status"].Value = "OnTaskFailed";
Dts.TaskResult = (int)ScriptResults.Success;
}

Select "OnError" - get Script Task – in the ReadWriteVariables option – select "User::Status" and this User::Status should be created in variable as "Status" and the value as "Completed".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
Dts.Variables["User::Status"].Value = "OnError";
Dts.TaskResult = (int)ScriptResults.Success;
}

Select "OnPostExecute" – get Script Task - In the ReadOnlyVariable option – select "System::SourceName,System::PackageName" and in the ReadWriteVariables option – select "User::IsPackageLevel" and this User::IsPackageLevel should be created in variable as "IsPackageLevel" and the value as "False".

And go to Edit Script option – write the below query

public void Main()
{
// TODO: Add your code here
if (Dts.Variables["System::SourceName"].Value.Equals(Dts.Variables["System::PackageName"].Value))
{
Dts.Variables["User::IsPackageLevel"].Value = true;
}
else
{
Dts.Variables["User::IsPackageLevel"].Value = false;
}
Dts.TaskResult = (int)ScriptResults.Success;
}

From the SRC IsPackageLevel – get a line to connect an Exe SQL task

"SQL Update [DB Name]_Audit "

In connector – select the Evalution Operation as "Expression"
and Value as "Success" and Expression as "@IsPackageLevel || @Status!="Completed"".

Also select the Multiple Constraints with "OR" option.

And connect line to – "SQL Update [DB Name]_Audit " (Exe SQL Task)

In which, do as below:

--write the below script to update values in Audit
UPDATE [dbo].[Audit]
SET [Status] = 'Completed'
,[EndTime] = GETDATE()
Where AuditId= 0
GO

Go to Expression – select the "SqlStatementSource" option and write below query:

"UPDATE [dbo].[Audit]
SET [Status] = '"+ @[User::Status] +"'
,[EndTime] = GETDATE()
Where AuditId= "+ (DT_WSTR, 20) (@[User::AuditId]) +"
GO
"