ODI 11g Reverse Engineering File modification - oracle-data-integrator

Sorry if this question is a bit too broad, I constantly use Reverse Engineering to read .TXT files in ODI 11g.
I was wondering if there is any way to modify or create an RKM(not sure if this is responsible) which by default assigns column Physical and Logical length to 300 for string datatype.
The default length which is assigned by ODI 11g is 50.
Is there any way to edit this?

You can do bulk changes of physical length and length with the next Groovy script.
Go to ODI > Tools > Groovy > New Script and copy paste the next Groovy code:
//Created by DI Studio
import ro.ns.odi.proxy.*
import oracle.odi.domain.model.*
import oracle.odi.domain.model.finder.*
import oracle.odi.domain.xrefs.expression.*
import oracle.odi.languages.support.*
IOdiEntityFactory odiFactory=OdiEntityFactory.createInstance(odiInstance)
IOdiBasicTemplate odiTemplate=odiFactory.newOdiTemplate()
String TABLE_NAME="SB_FINANCIALS_NEW_UU" //change with what you need
String MODEL_CODE="FILE" //change with what you need
odiTemplate.executeInTransaction{IOdiCommandContext ctx->
IOdiEntityManager odiManager=ctx.getSupportingOdiInstance().getTransactionalEntityManager()
IOdiDataStoreFinder dataStoreFinder=odiManager.getFinder(OdiDataStore)
OdiDataStore dataStore=dataStoreFinder.findByName(TABLE_NAME,MODEL_CODE)
assert dataStore!=null : "No data store was found. Please review the model code and data store name"
for (OdiColumn column:dataStore.columns){
println "Analyzing column ${column.name} with type ${column.dataType.name}, length ${column.getLength()} and scale ${column.scale}"
String dataTypeName=column.dataType.name
column.fileFieldDescriptor.bytes=4000 //change with what you need
//column.fileFieldDescriptor?.bytes=3000
column.length=4000 //change with what you need
column.dataType.name="String"
switch(dataTypeName){
case "String":
//column.scale=2
//column.fileFieldDescriptor?.decimalSeparator="x"
println "--Column Modified"
break
case "Numeric":
//column.scale=2
//column.fileFieldDescriptor?.decimalSeparator="x"
println "--Column Modified"
break
}
}
}
Read the Groovy code and change:
TABLE_NAME - name of datastore that you need to change the datatypes;
MODEL_CODE - code of MODEL from ODI > Models;
column.fileFieldDescriptor;
column.length;
column.dataType.

Related

Is thera a way to find forced execution context on datastore objects, somewhere in ODI metadata database?

I have a ODI 12c project with 30 mappings. I need to check if every "Component context" on every datastore object (source or target) is set to "Execution context" (not forced).
Is there a way to achive this by querying ODI underlying database so I don't have to do this manually, and to avoid possible mistakes ?
I have a list of ODI 12c Repository tables and comments on table columns which I got from the Oracle support website, and after hours of digging through database I still can't see this information stored in any table.
My package is located in SNP_PACKAGE, SNP_MAPPING has info about mapping , and SNP_MAP_COMP describes objects in mapping.
I have searched through many different tables as well.
A bit late but for anyone else looking
Messing about the tables is a no-no. APIs are better. Specially if you are to modify anything.
https://docs.oracle.com/en/middleware/data-integrator/12.2.1.3/odija/index.html
Run the following groovy script in ODI (Tools/Groovy/New Script). Should be simple enough to modify. Using the SDK gets a lot easier if you manage to set up a complete development env in IntelliJ or another Java IDE. Groovy in ODI opens up a whole new world.
//Created by DI Studio
import oracle.odi.domain.mapping.Mapping
import oracle.odi.domain.mapping.finder.IMappingFinder
tme = odiInstance.getTransactionalEntityManager()
IMappingFinder mapf = (IMappingFinder) tme.getFinder(Mapping.class)
Collection<Mapping> mappings = mapf.findByProject("PROJECT","FOLDER")
println("Found ${mappings.size()} mappings")
mappings.each { map ->
map.physicalDesigns.each{ phys ->
phys.physicalNodes.each{ node ->
println("${map.project.name}...${map.parentFolder.parentFolder?.name}.${map.parentFolder.name}.${map.name}.${phys.name}.${node.name}.defaultContext=${(node.context.defaultContext) ? "default" : node.context.name}")
}
}
}
It prints default or the set (forced) context. Seems forced context has been deprecated in 12c. Physical.node.context.defaultContext seems to mirror Component Context (Forced) in ODI Studio 12.2.1.3.
https://docs.oracle.com/en/middleware/data-integrator/12.2.1.3/odija/index.html
Update 2019-12-20 - including getExecutionContextName
The following script lists in a hierarchical manner and maybe easier to read the code. Not sure if you get what you are originally was after without having mapping with your exact setup.
//Created by DI Studio
import oracle.odi.domain.mapping.Mapping
import oracle.odi.domain.mapping.finder.IMappingFinder
import oracle.odi.domain.mapping.component.DatastoreComponent
tme = odiInstance.getTransactionalEntityManager()
String project = "PROJECT"
String parentFolder = "PARENT_FOLDER"
IMappingFinder mapf = (IMappingFinder) tme.getFinder(Mapping.class)
Collection<Mapping> mappings = mapf.findByProject(project, parentFolder)
println("Found ${mappings.size()} mappings")
println "Project: ${project}"
mappings.each { map ->
println "\tMapping: ..${map.parentFolder.parentFolder?.name}/${map.parentFolder.name}/${map.name}"
map.physicalDesigns.each{ phys ->
println "\t\tPhysical: ${phys.name}"
phys.physicalNodes.each{ node ->
println "\t\t\tNode: ${node.name}"
println "\t\t\t\tdefaultContext: ${(node.context.defaultContext)}"
println "\t\t\t\tNode context name: ${node.context.name}"
println "\t\t\t\tDatastoreComponent ExecutionContextName: ${DatastoreComponent.getDatastoreComponent(node)?.getExecutionContextName(node).toString()}"
}
}
}
Below is a list of some tables and columns that might hold the value you are looking for.
These tables and columns are from ODI 12.1.2, depending on the exact ODI version you are using, the structure could be a little different.
Here is also a query to retrieve this information directly from database.
-- Forced Contexts on Datastores in Mapping
SELECT MAPP.NAME MAP_NAME, MAPP_COMP.NAME DATASTORE_NAME,
MAPP_REF.QUALIFIED_NAME FORCE_CONTEXT
FROM SNP_MAPPING MAPP
INNER JOIN SNP_MAP_REF MAPP_REF
ON MAPP_REF.I_OWNER_MAPPING = MAPP.I_MAPPING
INNER JOIN SNP_MAP_PROP MAPP_PROP
ON MAPP_REF.I_MAP_REF = MAPP_PROP.I_PROP_XREF_VALUE
INNER JOIN ODIW12.SNP_MAP_COMP MAPP_COMP
ON MAPP_COMP.I_MAP_COMP = MAPP_PROP.I_MAP_COMP
WHERE
MAPP_REF.ADAPTER_INTF_TYPE = 'IContext'
and MAPP.NAME like %yourMapping%

SQL - trying to find a set of data which only has a certain set of values but not anything else in a few columns

Hopefully an easy problem for an experienced SQL person. I have an application which uses SQL Server, and I cannot perform this query in the application, so I'm hoping to back-door it, but I need help.
I have a table with a large list of emails and all its metadata. I'm trying to find email that is only between parties of this one company and flag them.
What I did was search where companyName.com is in To and From and marked a TagField as 1 (I did this through my application's front end).
Now what I need to do is search where any other possible values, ignoring companyName.com exist in To and From where I've already flagged them as 1 in TagField. From will usually just have one value, but To could have multiple, all formatted differently, but all separated by a semi-colon (I will probably have to apply this same search to CC and BCC columns, too).
Any thoughts?
Replace the ; with the empty string. Then check to see if the length changed. If there's one email address, there shouldn't be a ';'. You could also use the same technique to replace the company name with the empty string. Anything left would be the other companies.
select email_id, to_email
from yourtable
where TagField = 1 and len(to_email) <> len(replace(to_email,';',''))
This solution is based on the following thread
Number of times a particular character appears in a string
So I went an entirely different route and exported my data to a CSV and used Python to get to where I needed. Here's the code I used in case anybody needs it. What this returned for me was a list of DocIDs (unique identifiers that were in the CSV) where ever there was an email address in the To field that wasn't from one specific domain. I went into the original CSV and made sure all instances of this domain name were in all lowercase, too.
import csv
import tkinter as tk
from tkinter import filedialog
root = tk.Tk()
root.withdraw()
file_path = filedialog.askopenfilename()
sub = "domainname"
def findMultipleTo(dict):
for row in reader:
if row['To'].find(";") != -1:
toArray = row['To'].split(';')
newTo = [s for s in toArray if sub not in s]
row['To'] = newTo
else:
row['To'] = 'empty'
with open('location\\newCSV-BCCFieldSearch.csv', 'a') as f:
if row['To'] != "empty" and row['To'] != []:
print(row['DocID'], row['To'], file = f)
else:
pass
with open(file_path) as csvfile:
reader = csv.DictReader(csvfile)
findMultipleTo(reader)

STRING database Neo4j import

I have this network I'm working on from STRING database .
I have extracted that network so that I can impor it into Neo4j ,
The format I used is TSV and i have converted it to csv and done the import
yet , I can see only nodes and no relationships in between .
How can I make Neo4j able to know the relationships .and represent them
Here is the STRING network : http://string-db.org/cgi/network.pl?taskId=sPvqsEhi3Tk6&sessionId=MKQKZSH3dCb3&bottom_page_content=table
and that is the tabular form [1]: https://i.stack.imgur.com/IMMFk.png
And here is the Neo4j graph [1]: https://i.stack.imgur.com/eKeYv.png
I think the following will do the trick :
CREATE CONSTRAINT ON (n:Node) ASSERT a.NodeID IS UNIQUE;
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM 'file:///strings.csv'
AS line
WITH line
MERGE (n1:Node {NodeID: line.node1_string_internal_id, NodeName: line.node1})
MERGE (n2:Node {NodeID: line.node2_string_internal_id, NodeName: line.node2})
MERGE (n1)-[:INTERACTS {Score: TOFLOAT(line.combined_score)}]->(n2);
I hope this helps.
Regards,
Tom

how to connect MS Access database with matlab (transfer data from GUI and save in database )

Hello ppl I am trying to work with databases and I am new to Matlab.
I want to manipulate databeses created in MS Access but I don't know(I hope find a way to enter data from GUI (this GUI created using matlab ) and save in database)
I've designed the user interface in MATLAB, and create a database in MS Access
The problem I do not know how I connect between the database and MATLAB
I find some code to how connect between it.
dbpath = ['C:\Users\Esra\Documents\Esra.accdb'];
conurl = [['jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DSN='';DBQ='] dbpath];
con = database('','','','sun.jdbc.odbc.JdbcOdbcDriver', conurl);
I hope find good code or book about this .
final , i don not know if it is the correct place to my question or not , if not ,please put my question in correct place
You need to run SQL queries on the database; you can do this with database.fetch (and a few other friends).
The example query from the docs:
conn = database('dbtoolboxdemo','','');
setdbprefs('DataReturnFormat','cellarray')
results = fetch(conn, 'select productdescription from producttable')
% Not in the example in the docs: this syntax, which I prefer, is also supported
results = conn.fetch('select productdescription from producttable');
Note that you will also need to know how to write SQL. For that, there are plenty of resources online - you just have to search for them.

Extracting SQLite database into text fields from specific row

I am trying to connect to a SQLite database and have a method that specifies a specific row from the database (the first column in the database is “ID” and is a primary key) then extract the information from a few other columns in that row and display them in text fields.
This will be used for a simple Trivia game I am making; I will later make a random method that will choose the row at random.
I have been struggling with this problem for several weeks and I have been through loads of tutorials but all of them deal with displaying the data in a table view, I want to display it simply on text fields in a View based app. I am fairly confused at this point so any help starting from loading the database to displaying the data in the text fields would be GREATLY APPRECIATED!
Thanks!
Link to libsqlite3.dylib (and import <sqlite3.h>) to access the power of SQLite. There are a number of lightweight Objective-C front ends and I suggest you pick one. In this example, I use fmdb (https://github.com/ccgus/fmdb) to read the names of people out of a previously created database:
NSString* docsdir = [NSSearchPathForDirectoriesInDomains(
NSDocumentDirectory, NSUserDomainMask, YES) lastObject];
NSString* dbpath = [docsdir stringByAppendingPathComponent:#"people.db"];
FMDatabase* db = [FMDatabase databaseWithPath:dbpath];
if (![db open]) {
NSLog(#"Ooops");
return;
}
FMResultSet *rs = [db executeQuery:#"select * from people"];
while ([rs next]) {
NSLog(#"%# %#",
[rs stringForColumn:#"firstname"],
[rs stringForColumn:#"lastname"]);
}
[db close];
/* output:
Snidely Whiplash
Dudley Doright
*/
That illustrates talking to the database; knowing SQL is up to you (and is a different topic). You can include a previously constructed SQLite file in your app bundle, but you can't write to it there; the solution is to copy it from your app bundle into another location, such as the Documents directory, before you start working with it.
Finally, to put strings into text fields (UITextField), set their text property. So for example instead of the while loop shown above, where I log the database results, I could use those results to set text field values:
myTextField.text = [rs stringForColumn:#"firstname"];
myOtherTextField.text = [rs stringForColumn:#"lastname"];

Resources