I want to add all my failure logs to a column in DB.
As I am new to nigi. I'm confused on how to do that.
Need suggestion on how to do that.
You can follow the steps outlined below.
Under Controller Settings, add a REPORTING TASKS of type SiteToSiteBulletinReportingTask. This will capture bulletins. Make sure that Input Port Name property value is exact same as input port created at step 2. Configure rest of properties accordingly.
Add an input port at root level of NiFi i.e. not inside any process group.
When you start the reporting task, it will start sending bulletins to configured input port, i.e. NiFi instance will send data to itself, and you can further ingest into database.
Have a look at these implementation resources,
https://medium.com/#ashish211981/nifi-monitoring-driven-development-6b21046ee9e4
https://www.clearpeaks.com/health-monitoring-of-nifi-flows-using-bulletin-events/
Related
Let's say I run a recorded script for 'New User Registration' function of a web site to evaluate the response time for entire scenario. When I run the recorded script from JMeter, for each registration script, is there a new user record getting created in the application database ?
Yes, if you record registration and correlate it (meaning you create a valid unique name for every request) you will create a real user in your environment.
JMeter is simulating a real scenario which effect your environment.
That is part of the reason JMeter will be executed in different environment than production (as stage)
Well-behaved JMeter script must represent a real user using a real browser as close as it is possible.
Browsers execute HTTP requests and render the response
JMeter executes the same HTTP requests but doesn't render the response, instead it records performance metrics like response time, connect time, latency, throughput, etc.
HTTP is a stateful protocol therefore given you execute the same request you will get the same response. So if there are no mistakes in your script it either should create a new user or fail due to non-unique username error.
Yes, if your script accurately represents the full set of data flows associated with the business process, "New User Registration," then the end state of that process should be identical to that of the user behavior so modeled.
A record will be created in the database. If not, then your user is not accurate in its behavior
I have a python based client and server application. Client sends some data to the server based
on which server (running on python) runs some query on SQL SERVER database. For example let's consider client sends
a=5,b=10
Based on which Server computes :
#Server.py
import pyodbc
def test(a,b):
cnxn = pyodbc.connect(r'Driver={SQL Server};Server=.;Database=testdb;Trusted_Connection=yes;')
cursor = cnxn.cursor()
cursor.execute("select * from sample where a='"+a+"'" and b='"+b+"'"")
for row in cursor:
#doSomething ()
Both the variabes a and b are hardcoded inside the execute function.
Now what if I want to automate the whole process and by automate I mean to say what if
client sends c,d,e,f etc along with a,b. Do I have to change the server code every time or is there any
process to automate the whole server code irrespective of what client sends?
Thank You
It should have many solutions but depending on your architecture and the protocols between clients and the server.
For example,
We can define the message between clients and the server must follow below rules:
The message must be one json
The message must has one node named 'MessageType' or 'MessageID' or 'MessageName'
For your scenario, the message will be like
{'MessageType':'BeginSchedule', 'a':'test1', 'b':'test2', 'option1':1, 'option2':'t1', 'ClientID':'xxx'}
When the server receives one message from one client and its message type is 'BeginSchedule', then will invoke 'test(a,b)' to consume this request.
If there is another schedule, we can define one 'MessageType'='BeginSchedule2', then implement one handler in server code to consume any request whose MessageType='BeginSchedule2'.
The clients may send dirty data inside the message, you have to add some validators before calling the handler.
The clients may send extra data inside the message, you can design them as the optional arguments for the handler, please refer to:
Using Optional and Named Arguments
If you need your handlers accept infinite arguments, please refer to:
How to use *args and **kwargs in Python
In OSB Layer when the endpoint uri is changed, I need to alert the core group that the endpoint has changed and to review it. I tried SLA Alert rules but it does not have options for it. My question is, the endpoint uri should be saved somewhere in the underlying database. If so what is the schema and the table name to query it.
URI or in fact any other part of OSB artifact is not stored in relational database but rather kept in memory in it's original XML structure. It can be only accessed thru dedicated session management API. Interfaces you will need to use are part o com.bea.wli.sb.management.configuration and com.bea.wli.sb.management.query packages. Unfortunately it is not as straightforward as it sounds, in short, to extract URI information you will need to:
Create session instance(SessionManagementMBean)
Obtain ALSBConfigurationMBean instance that operates on SessionManagementMBean
Create Query object instance(BusinessServiceQuery) an run it on ALSBConfigurationMBean to get ref object to osb artifact of your interest
Invoke getServiceDefinition on your ref object to get XML service
definition
Extract URI from XML service definition with XPath
Downside of this approach is that you are basically pooling configuration each time you want to check if anything has changed.
More information including JAVA/WLST examples can be found in Oracle Fusion Middleware Java API Reference for Oracle Service Bus
There is also a good blog post describing OSB customization with WLST ALSB/OSB customization using WLST
The information about services and all its properties can be obtained via Java API. The API documentation contains sample code, so you can get it up and running quite quickly, see the Querying resources paragraph when following the given link.
We use the API to read the service (both proxy and business) configuration and for simple management.
As long as you only read the properties you do not need to handle management sessions. Once you change the values, you need to start a session and activate it once you are done -- a very similar approach to Service bus console.
I have a requirement to use OSB with DB Adapter as for given task, I believe BPEL Process will not be better performer than OSB. I have gone through some sites but all of them are demonstrating OEPE not OSB Console. Any response will be appreciated.
I have similar requirement in my project and I was wondering here and there. Here is final solution of the problem. I documented for you guys.
Steps in JDeveloper
1. Create DB Connection in JDeveloper.
2. Create a Generic Project.
3. Create Project as SOA project and use Empty Composite.
Steps to create DB Adaptor
Go to Composite Palette and choose AD Adaptor.
Drag and drop the DB Adaptor in the External Reference section (right side of Composite view).
It will open a wizard for selecting the connection and Other DB related process.
Give name to service that you want to create
Choose the DB Connection you have created already.
Move one with selecting your desired choice of Stored Procedure or Table (insert/Select) functionality that you want your web service to perform.
If you choose Table insert/Select then you can also choose the relation between tables as optional step.
Try to keep all steps default if you do not want any special features.
Once you finish all steps in wizard, then it will generate whole lot of files under your Project.
Look for 4 main files .jca, xsd/.xsd, *.mapping.xml, *.wsdl
If you see all these files generated well in your project then you are all set and we can move ahead. If not then look for the issue.
Steps in WLS Console
Create Data Source
1.Go to Left panel and choose Data Source.
2. Select New -- Generic Data Source
3. Give Name as you wish but for JNDI Name you have to use the connection name from the *.JCA file that you already have. Open the jca file and search for the ‘UIConnectionName’. This value will be used as JNDI name in data source creation.
4. Choose Driver as oracle.jdbc.xa.client.OracleXADataSource
5. Use the host and Port and other details that you had selected during DB Connection (in JDeveloper). This is obvious step.
6. Choose target AdminServer.
7. If you finish everything correctly. You are all set here and you can choose to test the connection and should be Test Success.
Configure the DB Adaptor
Go to Deployment on left hand panel.
In Deployed services. Search for DbAdaptor.
Click on the name DbAdaptor.
Select Configuration tab. And under it choose Outbound Connection Pool.
Here you will see (javax.resource.cci.ConnectionFactory)
Click on new.
Choose javax.resource.cci.ConnectionFactory
Next page will ask for JNDI Name. For this, you have to go to your *.jca file and choose the value which is already there for Connection-factory as Location (like connection-factory location=?). Use the value of Location as JNDI Name.
Transaction will be No Transaction. Choose this for now. This is need basis selection.
Leave other selection as is and then Finish.
Note : Once you finish. Go to Properties Tab again and look for a property name : xADataSourceName. Its value should be the same as UIConnectionName that you have in *.jca file. It should be populated. If not then you can set now as well. You have to double click on the value section and then write the name and press enter. Then Save it.
With above all steps we have set the DbAdaptor to point to same DB where we have our table/store procedure.
What we did above is:
• We Have JCA File and WSDL for our service. Along with DB Setting.
• We have a Data Source for our DB in WLS.
• We have the deployed DbAdaptor in WLS which is pointing to the same JNDI.
Steps in OSB Console
Create a Project.
Create 3 folders under Project. Like Resources, Proxy Service, Business Service.
Activate your project.
In Resources folder. We have to add JCA Mapping. Point it to the *.jca we already have from above steps (We had created it JDeveloper). Just give the path and Save. It may give you Error. Ignore for Now.
In Resources Folder. Add a WSDL and give the path of your *.wsdl. If it give Error. Then ignore for now.
In Resources Folder. Add a XSD and give path for the *.xsd we already have.
In Resources Folder. Add a XML and give path to the file *Mapping.xml.
Revisit resource WSDL again and click on its name. “Edit References”. Choose the XSD you already have and Save.
Revisit resource JCA and click on its name and edit reference and point it to the mapping.xml
By now you will see all errors gone.
Activate your project.
Go to Left hand side and select the Business Service and Create Business Service by selecting the wsdl project. On next page select Binding.
Then select JCA and give the ‘Location’ of IES (remember we have a value in connection-Factory in *.jca as Location. We had used it while creating DbAdaptor). Give that value here.
Move to next, next and next and finish.
Your business service is all set.
Now create a Proxy Service. Select Proxy Service and choose from B usiness Service option and do select the Business service you have create in above step.
Then click done
Activate your project.
That’s it we are all set. In proxy service you can click and see what is the End Point URl and keep it safe for your future reference. This is the url that we hav eto give on Client to call our service.
With this we all Done.
Call from Client.
Select SOAPUi for this purpose.
Create new project and give the end Point URL.
It will show all the services in it.
Select one service. Create Request.
Put the values in Request XML and Use the endpoint URL by appending your Localhost:port.
Click go and you will see the response back.
So, Finally : We are done with testing a OSB project with DB adaptor.
This looks long, but once you do it, its very easy for next services.
Thanks, Hope that will help.
The OSB Console does not support creation of the JCA DB Adapters, and neither does OEPE.
You need to create the DB Adapter in JDeveloper and then import it via OEPE/OSB Console. OEPE is easier ;-).
For a guide, see:
http://guidoschmutz.wordpress.com/2010/08/08/oracle-service-bus-11g-and-db-adapter-a-different-more-integrated-approach/
or
https://blogs.oracle.com/middleware/entry/using_jca_adapter_with_osb_11113
I'm working on a side project right now for an email client. I'm using a library to handle the retrieval of the messages from the server. However, I have a question on caching.
I don't want to fetch the entire list of headers everytime I load the client. Ideally, what I'd like to do is cache them and then update the list with what is on the server.
What's the best way to go about this? Should I store all the header information (including the server's message ID #) in a database, load the headers from that DB. Then as a background task sync up with the server...
Or is there a better way?
Look at the webmail sample of this open source project that use local caching:
http://mailsystem.codeplex.com/
If I remember well, he used a combination of local RFC822 plain text email storing with the message id as the filename and an index file with high level data.
Maybe the message itself where zipped to save disc space.
That's just a sample for the library, so don't expect code art there, but that's a start.