Table of contents
Working with the GenericWS (Describes the creation of: Send, Receive and Listener configurations)
This chapter gives an overview of the Generic Webservice Configurator (shortly GenericWS). The GenericWS is the product to create definitions to be used within a workflow. The GenericWS can function in multiple ways. We can divide the usage of the GenericWS into 3 parts:
- Send information to a webservice (consuming webservice) When you've got information which needs to be send to a webservice. Think of a target URL which hosts a database where you would like to send data to. You can create a WS action which consumes the target web service, retrieves information and store this in a Database (according to the WSDL structure), you can use this information to send calls to this target webservice.
- Retrieve information from a webservice (consuming webservice) If you would like to create a configuration to get information from another webservice. Think of a target URL which hosts information that you would like to retrieve/check (based on an interval). You can create a WS action which consumes the target web service, retrieves information and store this in a Database (according to the WSDL structure), you can use this information to retrieve information from this webservice.
- Listener (endpoint configuration) If you would like to create an endpoint and other web services need to post something to the DM platform. If you've got a webservice which can send information via web services, we can create a listener which hosts a structure (based on WSDL) to retrieve information on a certain port. This information can be used to be processed within the DM platform. It is stored in a database structure based on a WSDL.
Depending on the need you can create a configuration within the GenericWS. The GenericWS runs via the ESB Server. If the GenericWS doesn't start, or gives an error during the start, please check if all the ESB processes are running fine. After configuration of the GenericWS, it needs to be linked to a GenericWS Action in a workflow. The GenericWS actions are always called from an ESB workflow.
First select to create a New Soap or New REST call. If you've pressed the button, an auto name will be created, eg: NewInvocation_0102-1305. Change this to the appropriate name. This name will be used as a display name from within a Workflow. It's advisable to store the function in the name (consumer/endpoint). In this way you can easily find out which function the action has. Always click the Save button after you've modified something in the configuration.
REST Configuration (New Rest)
BETA: At the moment we're still working out configurations and testing this functionality. We already made it available for easy actions as FTP, and simple script execution. The more advanced options are still in beta and can be used as is. Options Tab The Rest functionality is based on a code library from Curl. Curl is a tool to easy call webservices from the cmd line. With this code library the we've extended our webservice functionallity enormous. Within the options screen all possible commands are shown. Every command is explained here. If the desired name/function is clicked, it can be configured. To configure a value, select it, check 'Setting value' and fill in the value. Common fields that need to be filled (with example values):
- URL https://eu3.salesforce.com/services/oauth2/token
- httpheader User-Agent: curl/7.37.1 Host: eu3.salesforce.com Connection: Keep-Alive
- timeout 20
- connectiontimeout 20
Processing Tab Main input/output is used to store the initial response values.
- DB Connection : Provide database name. The list contains an overview of available ESB database connection, found here: ..\S4M_DM\DMESB\Data\DbConnection\)
- Input table: The used input table. Please make sure to create this table first in MS SQL, also create the fields which are going to be used. Also add a mandatory field RowId first. Always start processing at 1. This RowId is used as a value to record the response value (when possible).E.g the fields 'user, password, user_token, client_id, client_secret' are used when you want to retrieve a SalesForce token. Create these fields and add RowId 1 with the needed values to be able to generate the token.
Output options (optional). With this configuration you can 'pass on' information and use this within a new call. This is mostly used if you retrieve a token first and you need the token to be provided for the following steps.
- Name: Name for the Output Options configuration
- Node/attribute to extract: value to extract from the main input/output table
- Output table: target table to be used
- Multi column result: **BETA**
- This will automatically attempt to create the columns in the target database table.
- Action on missing target fields: **BETA** (Currently Extend is the only working option)
- Result is single value
- Target field: Name of the field to be filled (make sure it exists)
- Target record filter (optional): define a filter for a record to be used. In our case the field is Code and the value is Jeroen
- Output options: **BETA**
- Write result as single value
- Result is JSON
- Result is column delimited (csv)
- Result is in XML
SOAP Configuration (New SOAP)
WebService config Tab Every function is explained. GenericWS has 3 types of configurations: Import/Send/Listeners. The fields needed for every configuration are marked after the description.
|WSDL||URL to WSDL to read, please provide the complete string including the ?WSDL partThe wsdl can also be based on a local file. Point to the file using the regular format eg: C:\test\my.wsdl||X||X||X|
|Username Password||Username and password to connect to the webservice, if needed.||X||X||X|
|Import or load assembly||When pressed the GenericWS will import the wsdl, the following fields will be automatically filled with options:-Webservice name-Webservice operation-EndpointAfter a successful import a .dll is generated and placed into the directory: C:\S4M_DM\DMESB\Bin\Agent\AssemblyCache\||X||X||X|
|Webservice name||Contains an overview of available webservice names||X||X||X|
|Webservice operation||Contains an overview of available webservice operation(s), every WS Action can make use of 1 operation||X||X||X|
|Enable listener host for WebService||Enable a listener based on the wsdl configuration||X|
|Listen address||Setup a listener, provide it in the following format (default it gets the wsdl listen address, this can be removed)Format:http://<DM Server NAME>:port/SpecificNameIt is possible to use 1 port and multiple listeners based on a /SpecificNameEg http://mydmserver:500/Test1 and http://mydmserver:500/Test2||X|
|Start listener automatically||Automatically start the listener when the ESB Server is started.||X|
|Enable WSDL||This enables the listener to provide and WSDL. When you go to the listen address and you add: /wsdl?singleWsdl or /wsdl?WSDL eg http://mydmserver:500/Test1/wsdl?SingleWSDL||X|
|Proxy||You can provide a proxy server.The address provided must use the following format http://<IPadress> e.g.: http://192.168.1.1.pac files are currently not supported.||X||X||X|
|Custom||This is used in specific configurations, can be empty by default. In case of ExamScheduler: When selected it keeps the session alive (based on username/password) provided.||X||X||X|
|Endpoint||Contains an overview of available Endpoints||X||X||X|
|DB Connection||The used (ESB based) database configuration. This is configured via the ESB. Place a configuration file in the C:\S4M_DM\DMESB\Data\DbConnection\(make a copy of the GenericWS.xml file, change the values, also the ID, restart DM after the file is saved in the directory)||X||X||X|
|No cascade delete||X||X||X|
|Param types||This is used in specific configurations, can be empty default. In case of ExamScheduler: Please look at the Exam Scheduler documentation for more information about the possible values (Objects)||X||X||X|
|File format||Ouput name for the log file. Make sure the file exists. TIP: This can be used with together with the ESB Trigger (file) functionality. If a record is logged, the Filtetrigger detects a change in the file and initiates and worflow.||X||X||X|
Execution Tab The execution tab is used to define the processing of the data. If the values are greyed out you need to check the first tab, one of the mandatory fields is missing then.
|Input/Output definition||Table prefix||Used prefix, make sure that it’s a short prefix otherwise the table name can get too long|
|Input/Output definition||Create mappings||Create needed table mappings (relations)|
|Input/Output definition||Optimize mappings||Optimize mappings (set index)|
|Input/Output definition||Create tables||Create the table (based on prefix, create and optimized mappings) for the webservice operation|
|Input/Output Cleanup||Delete Data||Delete all the data in all created tables|
|Input/Output Cleanup||Drop tables||Drop all tables|
|Listener||Start Listener||Manually start the listener If you exit the GenericWS configurator and you’ve started the listener, the port will stay active.|
|Listener||Stop Listener||Manually stop the listener|
|Output options||Log prefix||Table name used when ‘write structured output’ is unchecked and Log client messages is checked|
|Output options||Write structured output||Write output in structured target (under output definition defined) table(s)|
|Output options||Log client messages||Log all client messages. This can be used when data is not structured, but you would like to save the response value (to extract unstructured values)|
|Test action||Load input data||Manual load the input table|
|Test action||Prepare ws action||Manual prepare the action|
|Test action||Execute ws action||Manual execute the webservice action|
|Test action||Write output data||Manual write the output data to the target table|
|Log||Clear log||Clear GenericWS GUI Log|
|Log||Copy log||Copy the GenericWS GUI Log|
Working with the Generic Webservice Configurator
How to select the Webservice within a workflow? To add a Web service action:
- Open ESB/WFE
- Select an OIP Action from the Toolbox
- Set the Service to GenericWSExecute
- Within the parameters select and Agent and click WS Action
- Select the appropiate webservice action (created with the GenericWS Configurator)
Send information with the GenericWS (SOAP) This chapter will describe the data consolidation principle when using the web services to send data to a webservice. The consolidation is defined in the GenericWS and called from within the Workflow. When the workflow is called, the input data is send with the call. Via certain webservice methods it's possible to consolidate information from a ETL+ source to a target webservice. Overview: Follow these guidelines to create a consolidation transaction:
- Setup the appropiate transaction and configure this within the GenericWS (create the tables)
- Define the ETL+ Source tables/fields and prepare the consolidation to the GenericWS input tables
- Create an ETL+ Job (in ETL+ gui, tools -> scheduler) and schedule the table filling
- Create a WFE workflow and start with Source/Trigger action and add an OIP Action with the type GenericWS, select the created transaction from the first step
Retrieve information with the GenericWS (SOAP) This chapter describes the retrieval of webservice data in ETL+. Since this is based on webservice transaction, we first created an example retrieval webcall method: Sample configuration to retrieve Scientia ExamScheduler Locations: Sample data provided to the input table of the webservice: Important note: Since we only need output data from the webservice call, we have to provide the input table with the needed parameters and get the data in ETL+ by reading the output table after the webservice transaction. The output table names can be recognized from the prefix. The method name is defined according to the webservice structure, in this example the table used is: ES-Out-GetLocationConfigurations_HELocationConfigurationSnapshotDTO. This table contains all the locations. Below the numbers are explained more in detail:
- 1: Select a diagram. Select the GenericWS datasource and doubleclick on the blue bar on top of the diagram. The diagram will be opened.
- 2: To get data from (in our case) from Locations:
- On the left side, select tables, the white input field and type the defined prefix table name.
- The table with 'Results' at the end contains info regarding the webservice call execution status.
- Select the 'ES-Out-GetLocationConfigurations_HELocationConfigurationSnapshotDTO' table by clicking on it.
- 3: Under the general tab select the needed fields and press the buttons 'Import' -> 'View Last'
- 4: The desired fields and data is shown in the result window.
The data is now successfully imported and ready to be used within ETL+. If you're not familiar with the principles of ETL+, click here to get an overview of its features and posibilities.