I have created an SSIS package with a ForEach Loop which processes my files correctly. I am using a Data Flow Task to process the files. The problem is that I need to insert the Filename. I am using a Flat File Source. I right clicked and populated the FileNameColumnName property. It will only insert the first character of the filename string.
I have found plenty of post stating this should work without a hitch. That is not the case for me. I am at a loss. Any help would be appreciated.
I cannot use scripting.
↧
SSIS Package ForEach Loop Filename Returns First Character
↧
Microsoft patch and packages
Microsoft pushed a batch that adds a new column to CDC tables. We have tons of packages that reference these tables now with an extra column added.
Is there a easy way (BIML or other preferred) to update all the packages at once?
SQL Server 2016
↧
↧
Connect SQLLocalDB to SSIS
Hi,
How to connect to LocalDB v11.0 in SSIS 2016 ?
Please help.
Regards,
Natalia
↧
I am having troubles creating Derived Column SSIS
I am trying to create a derived column that will convert the date into a date keys, apparently, I got the expression right but when I run the package and go to my destination table I don't see my derived column created. I am not sure what could be the error, please help! ![alt text][1]
[1]: /storage/temp/4506-picture1.png
↧
Connection to Oracle
Hello! I have Oracle 12 installed on 64 bit machine. I get the connection error when trying to connect though SSIS package -- Test connection failed because of an error in initializing provider. Oracle client and networking components were not found. Is there a way to find what is missing.
Also I did verify the Oracle_Home environment variable exists.
Appreciate the help
Thank you
↧
↧
SSIS and SQL Database on the same machine?
I have a busy data warehouse running ETLs once an hour.
I would like to run my data warehouse database server and SSIS on the same server.
However I heard that the best practice is to separate Integration Service and database to different servers.
Is this correct?
Why?
Where can I find the official Microsoft information on the topic?
Thanks in advance
Alex
↧
How to avoid delays in Sql Server Agent Jobs every certain time?
Hi,
I have a SQL agent with 2 steps of SSIS packages and running every 10 seconds, every certain time the agent is taking longer than unusual, I don't if it is the database that is not allowing query constantly or an issue with the job.
I want to be able to get an average of 7-8 minutes which is acceptable.
Here is an image with the issue highlighted:
![Image][1]
[1]: /storage/temp/4562-issue.png
↧
Large Varchar(MAX) to DT_TEXT and Back (SSIS 2012)
My SSIS package needs to convert data from a varchar(max) field to DT_TEXT for processing (merge-join, conditional split, and insert or update depending on the split) and then back to varchar(max) in the destination table. However, it errors and won't just do what I tell it to - while the underlying data is just characters (not some weird BLOBs or some type of binary) it will not just push it through the pipe to the other end. I am also not, that I know of, using this specific field to do matching or conditional splitting.
Does anyone know any gotchas that I should be looking out for to have this flow through properly? Is there any way to do this? I am basically using a differential patter:
1. First data flow task: if source doesn't have an id, delete from destination
2. Second DFT - ignore any with date changed + ID match; if source date field + id exist, but destination doesn't, insert; if source date + id exist, but source is has a later change date than destination, update destination.
Hacks like converting to DT_STR 8000 characters won't work as this will lose data. While the data is just normal ASCII characters, I can't move it for anything.
I am sure I am not the only one with this pattern - how do people deal with it? I haven't seen any clearly worded explanations anywhere.
↧
how to stop/disable SSIS from logging to the event viewer application log
We run our SSIS packages using dtexec in a batch script and when the package fails it logs an error message in the Event viewer application log with event id of 12291. Is there a way to stop a failed SSIS package from logging a 12291 event code to the application log?
Thanks in advance for you help with this.
Glenn
↧
↧
Deleting rows from the destination that have been deleted from the source SSIS using Conditional Split
I'm building a SSIS package that Update, Insert and Delete rows from the destination based on the Source of course. Insert and update works fine but I'm not able to delete records from the destination that have been deleted from the source, as you can see in the image below I'm doing the test with an excel file and SQL server destination table. I don't know what exactly I have to put for the conditional split object in order to remove the old record from the destination. If you notice the records the has to be delete from the destination is highlighted in the Database Records image. Images below of the entire process step by step, other images are links:
![ETL][1]
![Conditional Split][2]
[Exce Source][3]
[OLE DB Command object for remove rows][4]
[Excel Records][5]
[Database Records][6]
[ETL Running][7]
[1]: /storage/temp/4575-etl.png
[2]: /storage/temp/4576-conditional-split.png
[3]: https://i.stack.imgur.com/rmzKn.png
[4]: https://i.stack.imgur.com/U8GRi.png
[5]: https://i.stack.imgur.com/DKIqT.png
[6]: https://i.stack.imgur.com/fPDA5.png
[7]: https://i.stack.imgur.com/h6tTl.png
↧
Datatype issue
Hello,
I am exporting a table from sql server to Excel by using SSIS(using oledb source and excel destination)
One of the column datatype is Bit (it should show result 1 when it is true and 0 when it is False) but in excel result I got -1 instead of 1.
I was totally confused and I even tried to convert that into INT which places an error in Excel destination task when I run the package
error: Column "Active" cannot convert between Unicode and non unicode string datatypes
Can someone help please
↧
How do I ensure SSISDB package and MSDB package are the same?
Hi everyone. I am new to package execution from external references, so this is basically a newbie question. Up until now, I have been deploying all of my packages from SSDT directly to SSISDB and then scheduling them to run from there in the SS Agent. I have a new requirement of executing packages from external references. It took me a while to figure out how to even get the package to MSDB. The only way that I was able to get a package deployed into MSDB was to connect to Integration Services in SSMS, right click to import a package, browse out to the package that I saved on the network, and add it. So my first question is: Is there a better, or more efficient way, to deploy my package to MSDB? My follow up question is: How do I ensure that the package that I have deployed to MSDB is exactly the same as the package that was deployed to SSISDB from Visual Studio? In this one case, I am sure because I am the only one working on this right now, but if we are deploying to two different locations with a disconnect in where the deploy is happening from, I can see that mistakes are bound to happen. Any recommendations on how you make sure MSDB and SSISDB stay in sync would be helpful. Thank you!
↧
Avoid providing Configuration file path in SSIS Package
I have created an SSIS Package with XML Configuration File
When I want to run on another server, I need to change the path of the configuration file in the Package file.
can I avoid in providing configuration file path in the package, please help?
↧
↧
avoid providing configuration file path in package
I have created a SSIS package with XML configuration file.
When I want to run in other server, I need to change the path of Configurationfile in Package file.
Can I avoid in providing configuration file pat in Package, please help
↧
SSIS, check excel destination if excel excel file exists
SSIS:
Hi, DO we have anything to do like (if exists) like checking the excel file in the folder, because the SSIS package should run every attempt
if I use drop and create(Execute sql task) for the first attempt it gonna fail as I don't have existing Excel file on my drive. Please let me know I struck thr :(
My requirement:
SQL server data should export to excel .
If Excel file did not exist, then Package should create a file, if it exists then the package should delete the file and should recreate
↧
SSIS flat file null dates
Hello! I am importing a flat file and if the date is null ,needs to be converted to a value 9999-99-99
The raw date is in the format 01-01-2016 and I used the expression
ISNULL([SDATE]) ? (DT_DBTIME)[SDATE] : (DT_DBTIME)(DT_STR,25,1252)"9999-99-99"
Retain null values is checked and in the advanced properties the output date is database timestamp
Appreciate any suggestions.
Thank you
↧
Exporting data from excel to sql server by using ssis and want to fill the empty cells with above value in process
Hi,
I am exporting data from Excel to SQL Server byh using SSIS and in column 1 there were some empty cells in excel. I want to fill those empty columns with above value
eg. TestID(column name)
test1
EmptyCell1
test2
Emptycell2
Here I want to fill EmptyCell1 with test1 and EmptyCell2 with test2 by SSIS process before inserting data into Table.
I can use TSQL update statement after inserting data into the table, but I don't want to do any changes after data insert.
Can anyone help?
↧
↧
Convert SSIS package to Powershell script
Hi,
I am a new to powershell scripts. I need to convert SSIS package (dtsx) to powershell. Can someone please point me to any tutorial or link that helps me convert. Thanks.
↧
ssis error
Currently there was some up gradation happened in our org from sql 2012 to sql 2016.So now we are upgrading our packages.All our packages are having source as excel.Earlier few of our packages were terminating abruptly when we were calling them from autosys while they execute successfully from local(through visual studio).We changed the DelayValidation to True and most of our packages are executing successfully except 2.The successfully executed packages were having some script task where we changed the delay validation property.But these 2 are failing abruptly when called from auosys but completes without any issues on local machine.The issue is we are not able to see any error messages in the log file and the log seems to be incomplete and every time it stops at a specific data flow without any error message.That data flow is having some basic tasks like insertion,lookup.I tried with Delayvalidation=true,MetadataValidation=False but nothing seems to be working.
Thanks in advance !
↧
How to choose a primary key for a fact table when the composite key is not unique
Hello everyone ,
My problem is : I 'm designing a data warehouse for a bank as my final year project using sql server data tools and sql server .
one of my data mart captures the income and expenses of the bank .
my fact table composed of these attributes :[PROD_CHARGE_ID],[CUSTOMER_ID],[CURRENCY_ID],[SECTOR_ID]
,[INDUSTRY_ID],[AGENCY_ID],[CATEGORY_ID],[COUNTRY_ID],[BOOKING_DATE_ID]
,[ACCOUNT_OFFICER],[RISK_CLASS],[COMPAGNY],[AMOUNT_FOREIGN],[AMOUNT_LOCAL]
The dimensions are : DimCustomer , DimCurrency,DimSector,DimIndustry,DimAgency,DimCategory , DimCountry and Dimdate .
The problem is that the combination of the Foreign key of dimensions doesn't allow me to have a unique composite primary key for the Fact table .
what's the solution in this case ? do I have to set an identity key for the Fact table ? or can I use the primary key of the table Source IncomeAndExpenses from where I extracted the measures and dimensions ?
↧