Hi all
Im having issues with an ssis package that processes change from a source db to a destination db. The package is set up to only process net changes and has multiple tables being loaded simultaneously.
The task sequence is 1) mark initial load start, 2) Mark initial load end, 3) Get processing range, 4) Mark Processing range. The package runs the initial load and the first incremental successfully (the first incremental is an update for rows that were added to source system whilst the initial package was loading). Then all subsequent incrementals fail as they try to insert duplicate rows into destination primary keys.
The cdc state variable sequence is;
ILSTART/IR/0x0000006E0000C4A00002//TS/2014-08-01T10:10:06.3579147/
ILEND/IR/0x0000006E0000C4A00002/0x0000006E0000C4B00002/TS/2014-08-01T10:11:39.0315573/
ILUPDATE/CS/0x0000006E0000C4A00002/CE/0x0000006E0000C4B00002/IR/0x0000006E0000C4A00002/0x0000006E0000C4B00002/TS/2014-08-01T10:12:05.8086365/
TFEND/CS/0x0000006E0000C4B00002/TS/2014-08-01T10:12:11.9693469/
TFSTART/CS/0x0000006E0000C4B00002/CE/0x00000F760000CFDF003D/TS/2014-08-01T10:26:54.5238988/
I cant understand how duplicate rows are making it through to the destination. Any ideas where I have gone wrong?
thanks in advance
↧
CDC SSIS package attempting to load duplicates
↧
Transfer Database Offline - network share file not found?
Hello,
I am trying to transfer a database offline between two servers. I have specified SourceDatabasefiles and DestinationDatabaseFiles. I created two folders and made them shared on both source(`\\EAPRDSQL41\Source_Share` is the shared folder on the source) and destination. I specified the "Network file Share" on both Source and Destination files.
But when I run the package, I get the error:
[Transfer Database Task] Error: The following exception occurred during an offline database transfer: System.IO.FileNotFoundException: Could not find file '\\EAPRDSQL41\Source_Share\SourceDatabase.mdf'.
File name: '\\EAPRDSQL41\Source_Share\SourceDatabase.mdf'
Why is the package looking for the mdf file located on the Network File share? Isn't the task supposed to create that file first?
Thank you!
↧
↧
What's the best way to get data from OLTP to OLAP system using SSIS?
hi,
I need to load data from OLTP system to OLAP system. i need to only load delta(new) data into OLAP system. which way is better?
1) Load data into stage table in OLAP system and then do lookup or Merge in SSIS.
2) Load data into stage table in OLAP system and kick off stored proc that does the lookup and merge.
OR is there any other option?
↧
Swapping partitions
hi! we have partitioning approach.
after the data is loaded to partitioned tables it is swapped to the main table.
I am getting the error:
Executing the query "declare @maxRows int declare @counter int declar..." failed with the following error: "ALTER TABLE SWITCH statement failed. index is in filegroup 'test' and partition 23 of index is in filegroup 'PRIMARY'
any guidance greatly appreciated.
thanks a lot
↧
How do you get SQL Agent to run SSIS packages
I am having a bear of time to get SQL Agent to run SSIS. The packages run manually with no problem. I have checked against others solutions including below from SQL Central. I put my AD domain acct in security, added the sysadmin role. Set up credential and proxy to run the job. I keep getting the message:
Unable to start execution of step 1 (reason: JobOwner HFBTNAD\slqintrg doesn't have permissions to use proxy 1 for subsystem SSIS). The step failed.**
This has to be something dorky for me to overlook. Any suggestions?
Run an SSIS Package Under a Different Account
By Polar Bear, 2010/03/25
↧
↧
SSIS Optimazation
I had an Backup server where I had two databases
I had created an sp which gets data from Database 1
I had created an ssis which uses Oledb source which uses the sp to get data from Database 1 and then transfer the data to Database 2 in a Table named Master using Oledb destination component
when I execute the sp in sql it takes only 9 seconds to get 436000 records (After clearing cache using DBCC FREEPROCCACHE) but when I runthe ssis to get the records and save it in table in another database it takes around 3 minutes or more
Can anybody please suggest a way on how we can optimize this or do it other way
↧
SSIS Insert and Update using Procedure
I had two databases on two different servers
In Database 1 there is a table called Rwamaster which stores the data and a flag which mentions whether it is an Insert or update or no action
I want to create an ssis which will transfer data from Rawmaster table of Database 1 to another table of same structure to Database 2 depending upon what is my flag that is insert or update
I don't want to use Oledb command as I found out that for every record it will make an roundtrip to server
Please suggets how to implement it in ssis
↧
Ftp task with auto date/date stamp
i have sFTP account to recieve this kind of files
/fcts_db_backup/fcts_backup 2014-08-22 02;01;24 (Full).zip
/fcts_db_backup/fcts_backup 2014-08-23 02;00;52 (Full).zip
/fcts_db_backup/fcts_backup 2014-08-24 02;01;44 (Full).zip
/fcts_db_backup/fcts_backup 2014-08-25 02;00;59 (Full).zip
They are Zip file that holds database backups. Every day i have to retrive it and restore it on my database.
there for,all i need is to find todays date/time stamp and put it on my local drive.
I tried to it with script task
NewDate = DateAdd("D", -i, newDate)
todaysDate = Format(newDate, "YYYY-MM-DD")
sFTP = "/fcts_db_Backup/fcts_Backup" & todaysDate & "*.Zip"
Dts.Variables("InputFile").Value = Destination
but when i connect it to the FTP Task and run it. it keeps saying
" FTP Task: File represented by "User::InputFile" does not exist"
The variable I give for InputFile is
"/fcts_db_Backup/fcts_Backup*.zip"
Please help me.
↧
Error code 0xC0202009 in SSIS
Hi, I created SSIS package which takes data from all production servers and insert that to repository. When I try to run package, it is running for few and then it is showing following error.
[OLE DB Source [61]] Error: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.
The AcquireConnection method call to the connection manager "Dynamic_SQLServerConn" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
↧
↧
How do I move multiple files of different category from one folder to different tables
There is a folder on my server in which 10-10 files are there for each category. I need to create a SSIS package which can automatically load the same type of category files to their corresponding tables. For example, for product information, I have 10 different files named product_1,product_2 etc. I need to fetch the data and put it into the [product] table in the database. Similarly I have 10 other categories. I want to do all this in one SSIS package.
↧
File System task in SSIS
Hello All, I am trying to create a SSIS Package which can
copy a previous day folder from one location to another using File System task
Please provide your valuable suggestion
Thank You Very Much
↧
SSIS dynamic excel tab
I have a sql table that I need to split into more 90 excel sheet based on a code. I could create an ssis package and use conditional split and create more than 90 excel sheet. But creating more than 90 excel sheet one at a time will be time consuming and if I have to use that package again for another table then I would have to make changes.
Is there an easier/faster way to achieve split a table into more than 90 excel tabs? Is it possible to use foreachloop and dynamically split and create excel tab?
↧
needs new metadata sas input
i have an ssis package to pull sas info into sql server. Sometimes new fields get added into sas datasets and this causes my ssis to fail.So, any ideas?
↧
↧
how to connect ssis connection in sql server?
when i'm trying to connect to ssis services in MSBI i got This Error Please Expalin
Connecting to the Integration Services service on the computer "JAI" failed with the following error: "Access is denied."
By default, only administrators have access to the Integration Services service. On Windows Vista and later, the process must be running with administrative privileges in order to connect to the Integration Services service. See the help topic for information on how to configure access to the service.
↧
SSIS Connection Manager
i have to pull data from XML file to 30 different Excel sheet.
Do i need to create 30 times connection Manager seperately for 30 different Excel Sheet.
Please suggest, is there any way to configure 30 different Excel Sheet with single Excel Configuration.
Thanks You Very Much
↧
sql loading data in diminternetsaletable to other table using ssispackages
Hi guys i have small doubt how can we move diminternetsales table to other table in adventure works using msbi(ssis) package which transformations i can use in this and how the process doing can u explain briefly guys
↧
Set up SSIS Logging with a Project Connection
Hi, I am trying to set up SSIS logging in SQL Server 2012 to write log entry events to a SQL Server database, however this creates an error at runtime if the connection is configured as a Project Connection:
**Error: The connection "Logging" is not found. This error is thrown by Connections collection when the specific connection element is not found.**
I would prefer to use a Project Connection as this can be used by other packages.
↧
↧
Is there a temp holding folder or table in SSIS?
I want to combine three tables from three different sources, which bring millions of rows from each table into one for daily use. Is there any way I can bring them into a temporary holding place using some task in SSIS so that the process won't consume more resources and will run faster, when combining into one table.
↧
0x80040E14 Description: "Invalid object name Error running SQL Agent Job but not directly in SSIS
I have created a package in BIDS. The package does the following:
1. Several data flow and SQL tasks to load text files into SQL tables. There is a connection manager for the SQL database (OLE DB Source object to connect to a SQL Server instance running on the local server).
2. A series of data flow tasks which
run a SQL Query against one or more of the SQL tables loaded in step 1
use a Data Conversion object to convert the DT_STR types to DT_WSTR, in order to prevent the errors regarding converting Unicode to non-Unicode formats.
Uses an Excel Destination object, linked to an Excel Connection Manager to map the columns provided by the Data conversion object to the correct columns in an Excel file and output the data to this Excel file.
The Excel file name is set using an expression for Connection Manager property, and is a .xls (Excel 97 - 2003) file. The Excel file is empty, apart from the column headings
2. The package runs successfully when executed in Visual Studio.
3. Add a 'wrapper' which uses configuration files to set the variable values and execute the package. The package is then stored in the MSDB.
4. There is a job which will run the .bat file and execute the package from the MSDB and send an email on success or failure.
This package and wrapper were working fine. I then:
1. Updated the package with logic for new tables and additional sheets in the Excel workbook (i.e. Report_2_b sheet is now Report_2_b_1, Report_2_b_2 etc. The package still works successfully in Visual Studio but fails when
Running the step to update the Report_2_b_1 sheet in the Excel workbook.
The two tables listed in the error message both exist prior to this task being run. The task has Delay Validation set.
the full text is posted below.
Step ID 1
Server ACMREPOSVR
Job Name NEQCA_TMP DSTHS ETL
Step Name 0 - Execute_ALL
Duration 00:08:58
Sql Severity 0
Sql Message ID 0
Operator Emailed
Operator Net sent
Operator Paged
Retries Attempted 0
Message
Microsoft (R) SQL Server Execute Package Utility
Version 10.50.2500.0 for 32-bit
Copyright (C) Microsoft Corporation 2010. All rights reserved.
Started: 2:26:52 PM
Error: 2014-10-09 14:27:24.99
Code: 0xC0202009
Source: Report 2b1 MED NEQCA_TMP_MED_NEW [1]
Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80040E14 Description: "Invalid object name 'dbo.NEQCA_TMP_MED_SVC_CAT_NEW'.".
End Error
Error: 2014-10-09 14:27:24.99
Code: 0xC004706B
Source: Report 2b1 MED SSIS.Pipeline
Description: "component "NEQCA_TMP_MED_NEW" (1)" failed validation and returned validation status "VS_ISBROKEN".
End Error
Error: 2014-10-09 14:27:24.99
Code: 0xC004700C
Source: Report 2b1 MED SSIS.Pipeline
Description: One or more component failed validation.
End Error
Error: 2014-10-09 14:27:24.99
Code: 0xC0024107
Source: Report 2b1 MED
Description: There were errors during task validation.
End Error
DTExec: The package execution returned DTSER_FAILURE (1).
Started: 2:26:52 PM
Finished: 2:35:49 PM
Elapsed: 537.547 seconds
-----Original Message-----
From: Database Reporting [mailto:Database@dovetailhealth.com]
Sent: Thursday, October 09, 2014 2:36 PM
To: DL Data Analytics
Subject: SQL Server Job System: 'NEQCA_TMP DSTHS ETL' completed on \\ACMREPOSVR
JOB RUN: 'NEQCA_TMP DSTHS ETL' was run on 10/9/2014 at 2:26:51 PM
DURATION: 0 hours, 8 minutes, 58 seconds
STATUS: Failed
MESSAGES: The job failed. The Job was invoked by User ACM\lpepi. The last step to run was step 1 (0 - Execute_ALL). The job was requested to start at step 1 (0 - Execute_ALL).
↧
Query for Chronological Order of SSIS Package
I have an SSIS package and I have enabled logging. I need to query the package and have it list all of the steps that are in the package. The sysssispackages table and the sysssislog is not very transaparent about how this could be done. has anyone here taken this on?
↧
