Showing posts with label packages. Show all posts
Showing posts with label packages. Show all posts

Wednesday, March 21, 2012

run SSIS package with ODBC connection via SQL agent

It seems there a lot of problems running SSIS packages under the sql agent. I have read the knowledgebase articles regarding permission issues etc but I still can't get my job to run. I can run any package as a job apart from a package that connects to an external database via an odbc connection. Has anyone had any luck with this and can let me in on the secret.

I had a lot of problems with this too, the way I was able to resolve it was to configure the SQL Agent to run as a domain service account, and then log into the server under the domain service account and create my SSIS package. Save the package to disk and then import it into SSIS, use the SQL Agent to run a job that points to the SSIS package and it then succeeded. In my case it was a security issue when the SQL Agent was running under a different account than the one I had created the package under. Once I used the same account to do everything I was able to get it to work.

|||

Hey Andy

Thanks for the reply, it seems that the issue is that I'm using a 64 bit OS and 32 bit odbc driver and when sql agent runs it uses the 64bit program files as opposed to the 32 bit program files which means that it couldn't find my odbc dsn. So to get around this had to create a batch file which calls the 'x86' program files.

Have also looked at getting a 64 bit odbc for my postgres database but seems that SSIS does point at 32 bit odbc connectiosn so don't see how to include the 64 bit odbc in SSIS.

Ugly and messy!

|||

Hi there,

Ahhhh yes, I tangled with this one too. What's happening is the SQL Management Studio, the Business Intelligence Studio, and SSIS are compiled as a 32 bit applications, the only 64 bit tool I have been able to find is the x64 DTS Wizard and it needs to be run from a command line or by double clicking it in Explorer. By using this you can connect to 64 bit ODBC drivers but you can't run any job as an SSIS package because there is no 64 bit SSIS. I was using an Oracle provider and found out it has a SQL Net issue with any path names like (x86), so I had to get a patch for that. So although you can install and run SQL 2005 on the x64 platform, 99% of the functionality is 32 bit.

Crazy stuff huh?

|||

I have an asp.net web application and a web service (both of them are created in VS 2005 - asp.net 2.0). They are located on the same web server. In both web.config files, I have set <authentication mode="Windows"/> and <identity impersonate="true"/>. Also, configured the IIS settings to use Integrated Windows Authentication and unchecked the Anonymous access (for both). The web service is called from the web app, so I have to pass credentials for authentication to the web service. The web service loads and executes a SSIS package. The package and all the other sql objects are located in the sql server 2005 (windows server 2003 - the same server as the web server).

When run the web service from develop environment (vs. 2005), I get whatever I expected. When call it from web application, however, the package failed (no error message).

From SSIS package log file, found that the user credentials (domain and username) were correctly passed from web service to sql server 2005 at the first event - packageStart, but the credentials (or operator) changed from domainABC\user123 to NT AUTHORITY\NETWORK SERVICE after packageStart. Then, it complains … either the user, domainABC\serverName$, does not have access to the database, or the database does not exist.

Please help and thank you all!

|||

Jenny,

It sounds like the credentials of your web application pool are being passed to SQL Server, the default account for the application pool is NT AUTHORITY\NETWORK SERVICE. Have you checked that?

|||

I had the same issue: A package using a 32-bit ODBC driver, it ran fine in my laptop but not in my 64-bit server. The work arround was to schedule it through an SQL Agent that used a command line to invoked the 32-bit version of dtsexec. The bottom line: the package runs in a 32-bit mode instead of 64-bit mode

Rafael

|||

Hi Andy, thanks for the response.

I have worked on something else these two days. I just checked the web application pool and both my web app and web service are pointed to the DefaultAppPool and its property setting for Identity as Predefined - Network Service. Should I create a new web application pool for both my web app and web service to use? I do need impersonation. How do I configure the new web application pool? Thanks!!!

|||

Here, are some more info.

In the SSIS package, there are three connection managers –

· A:Microsoft OLE DB Provider for Analysis Services 9.0 à connectionType=OleDbConnection

· B: .Net Providers \ SqlClient Data Provider à connection type=SqlConnection

· C:Native OLE DB \ Microsoft OLE DB Provider for SQL Server à connectionType=OLEDB

After ran the web application and check the sql database, I can tell that the package was reached and when through the first two steps which relate to the connection manager B – ADO.Net connection. The remaining steps failed which are related to the connection managers A & C. I think the credentials are passed ok but some setting related to the Analysis services are not correct. Any clues?

|||

Jenny,

There are a few different ways to go about this so why don't we try the easiest way first. According to the error message you are receiving, the domain\server$ does not have access to the database. So what's happening here is because your web application pool is running with the credentials of network service, SQL Server is viewing the account as ServerName$. So what you need to do is go into SQL Server>Security and add a new user with access permissions to the database. When you add the new user, type in the domain name and the server name with a $ at the end of it, for example: mydomain\myserver$

Make sure under the user mappings you check the box of the database that the user needs access to and also check the datareader box. This will give your web server access to read the database. If you need write permissions you can go back and check the datawriter box.

Typically you should run your web application under a domain service account, provide the domain service account access to the database, and use Windows authentication, something like this for your connection string: "server=MySQLServer; Integrated Security=SSPI; database=MyDatabase"

There are probably specific reasons why you want to use Impersonate, so give the mydomain\myserver$ example a try and see if you can get it to work.

|||

Hi Andy,

FYI: The problem was solved. What I did are list as below:

A. We created a new web app pool and pointed the both web application and web service to this web app pool (which was configured not use default identity - Predefined: Network Service but use Configurable - created a new use name).

B. Under SQL server , added this new user and assign the certain rights to it.

C. In the web application and web service, set impersonate=false (instead of true)

D. In the SSIS package, we also have a flat file destination connection which is to write the output to .txt file. I give the read & write rights to this new use. then, it works.

Thank you very much for your kind help!!!

Jenny

|||

Andy_T wrote:

Hi there,

Ahhhh yes, I tangled with this one too. What's happening is the SQL Management Studio, the Business Intelligence Studio, and SSIS are compiled as a 32 bit applications, the only 64 bit tool I have been able to find is the x64 DTS Wizard and it needs to be run from a command line or by double clicking it in Explorer. By using this you can connect to 64 bit ODBC drivers but you can't run any job as an SSIS package because there is no 64 bit SSIS. I was using an Oracle provider and found out it has a SQL Net issue with any path names like (x86), so I had to get a patch for that. So although you can install and run SQL 2005 on the x64 platform, 99% of the functionality is 32 bit.

Crazy stuff huh?

Had a somewhat different problem... BIDS is running 32 bit and connecting to 32-bit ODBC drivers (Paradox DB in this case), but when executed, the server keeps looking for 64-bit drivers. While I wish it would look for ALL possible drivers and automagically drop down from 64 to 32 if necessary, setting Run64BitRuntime to False fixed it for now (which is too bad, IBM has 64 bit drivers for the AS400). Haven't scheduled the package yet, though.|||Hi,

i'm having the same problem (trying to execute a package located on server A and a web service on server B), but i'm using IIS 5.1, so i think i don't have the web application pool feature.

I also have the package's Package source configured as SSIS Package Store so my path refers to "File System" source type.

from a custom application, i call the web service and i returns "failure" when trying to execute the package (this one is retrieved successfully, started but not executed).

Is there any way to solve this problem on a similar way like Jenny_99?

regards.

rodri|||

I had a similar issue and this is the workaround I found.

We installed SQL2005 with SSIS on a 64-bit windows 2003 server. An IBM ISeries for windows (V5R3M with the latest Service Pack) was used for connecting to AS400. There are actually two versions of ODBC drivers on the 64-bit windows (odbcad32.exe). One is located at Windows\System32. Another is at Windows\SysWOW64. Originally I thought the one in System 32 is 32-bit version, and the other one is 64-bit version. But the driver version number under SysWOW64 is actually consistent with the version number in our 32-bit windows 2003 server. So I am not so sure any more. Anyway, what I found out is that DSN under SysWOW64 is actually used when the package is running in BIDS, or SSIS on the server. The one under System32 is used when the package is scheduled as a job. So I created the same DSN on both drivers to cover all bases since I don't know how to force the SQL agent to use a specific driver.

run SSIS package with ODBC connection via SQL agent

It seems there a lot of problems running SSIS packages under the sql agent. I have read the knowledgebase articles regarding permission issues etc but I still can't get my job to run. I can run any package as a job apart from a package that connects to an external database via an odbc connection. Has anyone had any luck with this and can let me in on the secret.

I had a lot of problems with this too, the way I was able to resolve it was to configure the SQL Agent to run as a domain service account, and then log into the server under the domain service account and create my SSIS package. Save the package to disk and then import it into SSIS, use the SQL Agent to run a job that points to the SSIS package and it then succeeded. In my case it was a security issue when the SQL Agent was running under a different account than the one I had created the package under. Once I used the same account to do everything I was able to get it to work.

|||

Hey Andy

Thanks for the reply, it seems that the issue is that I'm using a 64 bit OS and 32 bit odbc driver and when sql agent runs it uses the 64bit program files as opposed to the 32 bit program files which means that it couldn't find my odbc dsn. So to get around this had to create a batch file which calls the 'x86' program files.

Have also looked at getting a 64 bit odbc for my postgres database but seems that SSIS does point at 32 bit odbc connectiosn so don't see how to include the 64 bit odbc in SSIS.

Ugly and messy!

|||

Hi there,

Ahhhh yes, I tangled with this one too. What's happening is the SQL Management Studio, the Business Intelligence Studio, and SSIS are compiled as a 32 bit applications, the only 64 bit tool I have been able to find is the x64 DTS Wizard and it needs to be run from a command line or by double clicking it in Explorer. By using this you can connect to 64 bit ODBC drivers but you can't run any job as an SSIS package because there is no 64 bit SSIS. I was using an Oracle provider and found out it has a SQL Net issue with any path names like (x86), so I had to get a patch for that. So although you can install and run SQL 2005 on the x64 platform, 99% of the functionality is 32 bit.

Crazy stuff huh?

|||

I have an asp.net web application and a web service (both of them are created in VS 2005 - asp.net 2.0). They are located on the same web server. In both web.config files, I have set <authentication mode="Windows"/> and <identity impersonate="true"/>. Also, configured the IIS settings to use Integrated Windows Authentication and unchecked the Anonymous access (for both). The web service is called from the web app, so I have to pass credentials for authentication to the web service. The web service loads and executes a SSIS package. The package and all the other sql objects are located in the sql server 2005 (windows server 2003 - the same server as the web server).

When run the web service from develop environment (vs. 2005), I get whatever I expected. When call it from web application, however, the package failed (no error message).

From SSIS package log file, found that the user credentials (domain and username) were correctly passed from web service to sql server 2005 at the first event - packageStart, but the credentials (or operator) changed from domainABC\user123 to NT AUTHORITY\NETWORK SERVICE after packageStart. Then, it complains … either the user, domainABC\serverName$, does not have access to the database, or the database does not exist.

Please help and thank you all!

|||

Jenny,

It sounds like the credentials of your web application pool are being passed to SQL Server, the default account for the application pool is NT AUTHORITY\NETWORK SERVICE. Have you checked that?

|||

I had the same issue: A package using a 32-bit ODBC driver, it ran fine in my laptop but not in my 64-bit server. The work arround was to schedule it through an SQL Agent that used a command line to invoked the 32-bit version of dtsexec. The bottom line: the package runs in a 32-bit mode instead of 64-bit mode

Rafael

|||

Hi Andy, thanks for the response.

I have worked on something else these two days. I just checked the web application pool and both my web app and web service are pointed to the DefaultAppPool and its property setting for Identity as Predefined - Network Service. Should I create a new web application pool for both my web app and web service to use? I do need impersonation. How do I configure the new web application pool? Thanks!!!

|||

Here, are some more info.

In the SSIS package, there are three connection managers –

· A:Microsoft OLE DB Provider for Analysis Services 9.0 à connectionType=OleDbConnection

· B: .Net Providers \ SqlClient Data Provider à connection type=SqlConnection

· C:Native OLE DB \ Microsoft OLE DB Provider for SQL Server à connectionType=OLEDB

After ran the web application and check the sql database, I can tell that the package was reached and when through the first two steps which relate to the connection manager B – ADO.Net connection. The remaining steps failed which are related to the connection managers A & C. I think the credentials are passed ok but some setting related to the Analysis services are not correct. Any clues?

|||

Jenny,

There are a few different ways to go about this so why don't we try the easiest way first. According to the error message you are receiving, the domain\server$ does not have access to the database. So what's happening here is because your web application pool is running with the credentials of network service, SQL Server is viewing the account as ServerName$. So what you need to do is go into SQL Server>Security and add a new user with access permissions to the database. When you add the new user, type in the domain name and the server name with a $ at the end of it, for example: mydomain\myserver$

Make sure under the user mappings you check the box of the database that the user needs access to and also check the datareader box. This will give your web server access to read the database. If you need write permissions you can go back and check the datawriter box.

Typically you should run your web application under a domain service account, provide the domain service account access to the database, and use Windows authentication, something like this for your connection string: "server=MySQLServer; Integrated Security=SSPI; database=MyDatabase"

There are probably specific reasons why you want to use Impersonate, so give the mydomain\myserver$ example a try and see if you can get it to work.

|||

Hi Andy,

FYI: The problem was solved. What I did are list as below:

A. We created a new web app pool and pointed the both web application and web service to this web app pool (which was configured not use default identity - Predefined: Network Service but use Configurable - created a new use name).

B. Under SQL server , added this new user and assign the certain rights to it.

C. In the web application and web service, set impersonate=false (instead of true)

D. In the SSIS package, we also have a flat file destination connection which is to write the output to .txt file. I give the read & write rights to this new use. then, it works.

Thank you very much for your kind help!!!

Jenny

|||

Andy_T wrote:

Hi there,

Ahhhh yes, I tangled with this one too. What's happening is the SQL Management Studio, the Business Intelligence Studio, and SSIS are compiled as a 32 bit applications, the only 64 bit tool I have been able to find is the x64 DTS Wizard and it needs to be run from a command line or by double clicking it in Explorer. By using this you can connect to 64 bit ODBC drivers but you can't run any job as an SSIS package because there is no 64 bit SSIS. I was using an Oracle provider and found out it has a SQL Net issue with any path names like (x86), so I had to get a patch for that. So although you can install and run SQL 2005 on the x64 platform, 99% of the functionality is 32 bit.

Crazy stuff huh?

Had a somewhat different problem... BIDS is running 32 bit and connecting to 32-bit ODBC drivers (Paradox DB in this case), but when executed, the server keeps looking for 64-bit drivers. While I wish it would look for ALL possible drivers and automagically drop down from 64 to 32 if necessary, setting Run64BitRuntime to False fixed it for now (which is too bad, IBM has 64 bit drivers for the AS400). Haven't scheduled the package yet, though.|||Hi,

i'm having the same problem (trying to execute a package located on server A and a web service on server B), but i'm using IIS 5.1, so i think i don't have the web application pool feature.

I also have the package's Package source configured as SSIS Package Store so my path refers to "File System" source type.

from a custom application, i call the web service and i returns "failure" when trying to execute the package (this one is retrieved successfully, started but not executed).

Is there any way to solve this problem on a similar way like Jenny_99?

regards.

rodri|||

I had a similar issue and this is the workaround I found.

We installed SQL2005 with SSIS on a 64-bit windows 2003 server. An IBM ISeries for windows (V5R3M with the latest Service Pack) was used for connecting to AS400. There are actually two versions of ODBC drivers on the 64-bit windows (odbcad32.exe). One is located at Windows\System32. Another is at Windows\SysWOW64. Originally I thought the one in System 32 is 32-bit version, and the other one is 64-bit version. But the driver version number under SysWOW64 is actually consistent with the version number in our 32-bit windows 2003 server. So I am not so sure any more. Anyway, what I found out is that DSN under SysWOW64 is actually used when the package is running in BIDS, or SSIS on the server. The one under System32 is used when the package is scheduled as a job. So I created the same DSN on both drivers to cover all bases since I don't know how to force the SQL agent to use a specific driver.

sql

Monday, March 12, 2012

run packages as part of procedure

I have created SQL code that is contained in different procedures. I have a procedure that then runs all of the pieces that I wish to run. My last step in my process is to export the data to my c: drive. During this process I created a package to be rerun. How do I include the rerunning of my package automatically as part of the procedural process?

How about creating a job for the package, and calling sp_start_job from your procedure. Note this will not wait for the package to complete.

A less attactive option is to enable xp_cmdshell, and call DTEXEC through xp_cmdshell, which itself can be called from your procedure.

Run multiple SSIS packages in order

I built 3 diffrent packages and i want to be executed in order. The first one is exporting some tables to another database and the other two packages are based on the database built in the first package.

Any suggestions?

Thank you in advance:)

I assume you wish to automate this process. The simplest option would be to create a SQL Server Agent job, and just have three steps within the job, one for each package.

A more SSIS orientated approach would be to use a master package, that has three execute package tasks, one for each of your existing packages. Link the Execute package Tasks together with workflow constraints to enforce the order of execution for the tasks and hence the packages.

|||

Darren,

Just a question: when we're talking about the maximum number of DTSX inside a DTSPROJ only limitation is the memory?

TIA

|||

Yes.

If you do have a lot then I'd be tempted to write a more dynamic process.

Perhaps use a SQL table to contain the list of packages, and also have a column for the order. You could then use the Exec SQL Task with simple SELECT query to extract the list of packages in order of execution. You could store this results in a variable, and then loop and shred this via the For Each Loop.

This example uses Data Flow task to get the result set into a variable, but I normally find an Exec SQL Task easier. It does show the shredding of the results via the loop though.

Shredding a Recordset

(http://www.sqlis.com/default.aspx?59)

|||

I used the SQL Server Agent Job.

thanks for the help :)

Friday, March 9, 2012

Run DTS packages from a structured storage file instead SQL Server

I'm wondering if this is a good idea. Can anybody list me the pros and cons
of each? We have someone pushing to run them from a file instead of a
package in SQL Server and I want to make sure there aren't any issues.
Thanks,
VanVan,
Using a structured storage file (.dts) increases the flexability of how and
where DTS packages are executed as well as increases the portability of DTS
packages. Security may be a concern using structured storage files.
HTH
Jerry
"Van" <Van@.discussions.microsoft.com> wrote in message
news:B3F75D14-7476-4DE1-A48C-A32F658F7B76@.microsoft.com...
> I'm wondering if this is a good idea. Can anybody list me the pros and
> cons
> of each? We have someone pushing to run them from a file instead of a
> package in SQL Server and I want to make sure there aren't any issues.
> Thanks,
> Van

Run DTS packages

I have users who need to run DTS packages but I don't want to put the full
SQL toolset on their machine. Is there a way to just install the ability to
run a DTS package?Take a look at
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dtssql/dts_tools_other_8reb.asp
I think you can simply install client connectivity and dtsrun.exe... You
can even encrypt the connection information so they will not even know the
login stuff.
Also take a look at DTSRUNui...which will give your end user a little gui
interface to run packages.
--
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Mark Polino" <mpolino@.earthlink.net> wrote in message
news:uifNMo9VEHA.3696@.TK2MSFTNGP10.phx.gbl...
> I have users who need to run DTS packages but I don't want to put the full
> SQL toolset on their machine. Is there a way to just install the ability
to
> run a DTS package?
>
>|||There are dts specific resource files as well as dlls that
need to be registered for users to be able to execute
packages. You can find the list of these in the following
article:
Redistributing DTS with your program
http://www.sqldts.com/default.aspx?225
-Sue
On Mon, 21 Jun 2004 17:54:02 -0400, "Mark Polino"
<mpolino@.earthlink.net> wrote:
>I have users who need to run DTS packages but I don't want to put the full
>SQL toolset on their machine. Is there a way to just install the ability to
>run a DTS package?
>

Run DTS packages

I have users who need to run DTS packages but I don't want to put the full
SQL toolset on their machine. Is there a way to just install the ability to
run a DTS package?
Take a look at
http://msdn.microsoft.com/library/de...other_8reb.asp
I think you can simply install client connectivity and dtsrun.exe... You
can even encrypt the connection information so they will not even know the
login stuff.
Also take a look at DTSRUNui...which will give your end user a little gui
interface to run packages.
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Mark Polino" <mpolino@.earthlink.net> wrote in message
news:uifNMo9VEHA.3696@.TK2MSFTNGP10.phx.gbl...
> I have users who need to run DTS packages but I don't want to put the full
> SQL toolset on their machine. Is there a way to just install the ability
to
> run a DTS package?
>
>
|||There are dts specific resource files as well as dlls that
need to be registered for users to be able to execute
packages. You can find the list of these in the following
article:
Redistributing DTS with your program
http://www.sqldts.com/default.aspx?225
-Sue
On Mon, 21 Jun 2004 17:54:02 -0400, "Mark Polino"
<mpolino@.earthlink.net> wrote:

>I have users who need to run DTS packages but I don't want to put the full
>SQL toolset on their machine. Is there a way to just install the ability to
>run a DTS package?
>

Run DTS packages

I have users who need to run DTS packages but I don't want to put the full
SQL toolset on their machine. Is there a way to just install the ability to
run a DTS package?Take a look at
other_8reb.asp" target="_blank">http://msdn.microsoft.com/library/d...
other_8reb.asp
I think you can simply install client connectivity and dtsrun.exe... You
can even encrypt the connection information so they will not even know the
login stuff.
Also take a look at DTSRUNui...which will give your end user a little gui
interface to run packages.
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Mark Polino" <mpolino@.earthlink.net> wrote in message
news:uifNMo9VEHA.3696@.TK2MSFTNGP10.phx.gbl...
> I have users who need to run DTS packages but I don't want to put the full
> SQL toolset on their machine. Is there a way to just install the ability
to
> run a DTS package?
>
>|||There are dts specific resource files as well as dlls that
need to be registered for users to be able to execute
packages. You can find the list of these in the following
article:
Redistributing DTS with your program
http://www.sqldts.com/default.aspx?225
-Sue
On Mon, 21 Jun 2004 17:54:02 -0400, "Mark Polino"
<mpolino@.earthlink.net> wrote:

>I have users who need to run DTS packages but I don't want to put the full
>SQL toolset on their machine. Is there a way to just install the ability to
>run a DTS package?
>

Run DTS package in sql server 2005

We are about to migrate a 2000 server to a different server with 2005. One of the dts packages has been giving us problems converting it, so we want to leave it as a dts for now so we can look at it later but still upgrade now. Since I'm moving the database to a new server I need to know how to set up a job to run the dts, as well as how to ensure the dts is moved to the new server as well.

my initial thought is just to export the dts then import it to the new server. After that set up a new job on the new server to run the dts. I cant figure out how to set up the job though. I guess I need to write a dtsrun command, but I've always done that by right clicking on the dts and having the wizard create the command for me. And it seems with 2005 i can't do that even with the backward compantability tools installed. Anyone can help?
Yes the legacy DTS packages still works and you can still use them with SQL Server 2005 / Calling them inside a JOB using DTSRUN. Install SQL Server 2005 DTS Designer Components
for legacy support and continue working with legacy 2000 DTS packages

HTH
Vishal

Wednesday, March 7, 2012

Run a SSIS Package including Windows Tasks Scheduler?

Hello

As a beginner I have created some Packages to import Data to SQL Server.

The Packages are running well.

I have created a Task to run the Package on a SQL SERVER 2005 Express. The Task starts.

Then a Execute Package Wizard Picture appears on the Screen.

I have mannually to select the Connection Manager and the Connection String.

After the selection the Task runs without any Problem.

Now I like to get the Task to run without any manual intervention.

I was looking in Internet but could not find a solution.

Can someone give me a hint how to get the Task working without intervention?

Thanks for an early answer.

Regards

Chaepp

Specify dtexec.exe as executable to run in the Job definition.
See Books Online for dtexec command line parameters reference. In simplest case: "dtexec.exe /f c:\package.dtsx"|||

Hello Michael

I'ts working now. Thank you very much for your help!

Regards

Chaepp