The Xtract Universal server allows accessing meta data and logging information through web calls.
Note: This article uses the syntax of the Xtract Universal Webserver before version 6.0.0 When using Xtract Universal version 6.0.0 or higher, refer to Online Help: Log Access via Web Service.
Basic URL #
The basic URL for web calls uses the following format: [protocol]://[host or IP address]:[port]/.
Make sure to use the correct protocol:
| Protocol | Syntax | Example |
|---|---|---|
| HTTP | http://[host].[domain]:[port] |
http://todd.theobald.local:8065 |
| HTTP | http://[host]:[port] |
http://localhost:8065 |
| HTTPS | https://[host].[domain]:[port] |
https://todd.theobald.local:8165Requires a dedicated host name and X.509 certificate, see web server settings. |
Note: Make sure to use the correct ports, see Server Ports.
Web calls can be used to:
- Query all Logs
- Query Logs at Specific Timestamps
- Query all Server Logs
- Query Server Logs at Specific Timestamps
- Query a List of all Defined Extractions
- Query a Specific Extraction at a Specific Timestamp
- Query the Extraction Status
- Query metadata of extractions, see Metadata access via HTTP-JSON
- Run extractions, see Execute and Automate - Call via Webservice
Query all Logs #
| URL | Description |
|---|---|
http(s)://[host]:[port]/log/?req_type=all |
Returns all extraction and server logs. |
Note: Server logs files are deleted after a defined period of days, see Server Setting - Web Server.
Response
The web call returns the following information:
- LineCount: row number of the log entry.
- Name: name of the extraction / name of the server.
- Timestamp: timestamp of the extraction or server log.
- State: returns a number between 2 and 4 for a server extraction or the number 5 for a server log, see table below.
- StateDescr: description of the state, see table below.
- LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
- Source technical name of the component that generates the log info.
- Message: content of the log.
| State | StateDescr | Description |
|---|---|---|
| 2 | Running | The extraction is running. |
| 3 | FinishedNoErrors | Extraction succeeded without errors. |
| 4 | FinishedErrors | Extraction is finished with at least one error. |
| 5 | NotAvailable | The status for a server log. |
For information on how to read extractions logs, see Extraction Logs.
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/log/?req_type=all |
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message 1,MAKT,2023-02-20_09:49:23.941,3,FinishedNoErrors,Info,LiveDataExtraction,Product version 5.21.10.14 2,MAKT,2023-02-20_09:49:23.942,3,FinishedNoErrors,Info,LiveDataExtraction,Using Theobald.Extractors Interface 3,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Theobald.Extractors version 1.39.3.13 4,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Executing Table extraction ... 1,[server],2023-02-20_09:49:10.208,5,NotAvailable,Warning,VersionStore,Configuration was created by a development build. This can lead to unexpected behaviour. 2,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Trying to listen on [::]:8065... 3,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Listening on [::]:8065 4,[server],2023-02-20_09:49:23.353,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51531 connected ... |
Query Logs at Specific Timestamps #
| URL | Description |
|---|---|
http(s)://[host]:[port]/log/?req_type=all×tamp=[yyyy-MM-dd] |
Returns all logs of the specified date. |
http(s)://[host]:[port]/log/?req_type=all×tamp=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns all logs of the specified timestamp. |
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd] |
Returns all logs after the specified date. |
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns all logs after the specified timestamp. |
http(s)://[host]:[port]/log/?req_type=all&max=[yyyy-MM-dd] |
Returns all logs before the specified date. |
http(s)://[host]:[port]/log/?req_type=all&max=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns all logs before the specified timestamp. |
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd]&max=[yyyy-MM-dd] |
Returns all logs between two dates. |
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd_HH:mm:ss.SSS]&max=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns all logs between two timestamps. |
http(s)://[host]:[port]/log/?req_type=all&past_days=[number_of_days] |
Returns all logs since n days. |
Response
The web call returns the following information:
- LineCount: row number of the log entry.
- Name: name of the extraction / name of the server.
- Timestamp: timestamp of the extraction or server log.
- State: returns a number between 2 and 4 for a server extraction or the number 5 for a server log, see table below.
- StateDescr: description of the state, see table below.
- LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
- Source technical name of the component that generates the log info.
- Message: content of the log.
| State | StateDescr | Description |
|---|---|---|
| 2 | Running | The extraction is running. |
| 3 | FinishedNoErrors | Extraction succeeded without errors. |
| 4 | FinishedErrors | Extraction is finished with at least one error. |
| 5 | NotAvailable | The status for a server log. |
For information on how to read extractions logs, see Reading Extraction Logs.
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/log/?req_type=all&past_days=2 |
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message 1,MAKT,2023-02-20_09:49:23.941,3,FinishedNoErrors,Info,LiveDataExtraction,Product version 5.21.10.14 2,MAKT,2023-02-20_09:49:23.942,3,FinishedNoErrors,Info,LiveDataExtraction,Using Theobald.Extractors Interface 3,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Theobald.Extractors version 1.39.3.13 4,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Executing Table extraction ... 1,[server],2023-02-20_09:49:10.208,5,NotAvailable,Warning,VersionStore,Configuration was created by a development build. This can lead to unexpected behaviour. 2,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Trying to listen on [::]:8065... 3,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Listening on [::]:8065 4,[server],2023-02-20_09:49:23.353,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51531 connected ... |
https://todd.theobald.local:8165/log/?req_type=all&min=2023-02-20_09:49:24.500&max=2023-02-20_09:50:00.000 |
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message 22,MAKT,2023-02-20_09:49:24.500,3,FinishedNoErrors,Debug,TheoReadTableExtractor,Data will be extracted in dialog work process 23,MAKT,2023-02-20_09:49:24.501,3,FinishedNoErrors,Debug,TheoReadTableExtractor,"Fetching packages (50,000 rows per package)" 24,MAKT,2023-02-20_09:49:24.653,3,FinishedNoErrors,Debug,TheoReadTableExtractor,Z_THEO_READ_TABLE version 1.x 25,MAKT,2023-02-20_09:49:24.653,3,FinishedNoErrors,Debug,TheoReadTableExtractor,Received package #1 (1 rows) 26,MAKT,2023-02-20_09:49:24.657,3,FinishedNoErrors,Info,LiveDataExtraction,Starting to write 1 rows to destination... 27,MAKT,2023-02-20_09:49:24.668,3,FinishedNoErrors,Info,LiveDataExtraction,Finished writing rows to destination 28,MAKT,2023-02-20_09:49:24.712,3,FinishedNoErrors,Info,TheoReadTableExtractor,Extraction finished - received 1 rows in total 29,MAKT,2023-02-20_09:49:24.714,3,FinishedNoErrors,Debug,LiveDataExtraction,Writing results to destination completed 6,[server],2023-02-20_09:49:24.802,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16240) exited with 0x0 - The operation completed successfully 7,[server],2023-02-20_09:49:36.257,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51533 connected 8,[server],2023-02-20_09:49:36.262,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16368) started |
Query all Server Logs #
| URL | Description |
|---|---|
http(s)://[host]:[port]/log/?req_type=server |
Returns a list of timestamps that correspond to server logs. |
Response
The web call returns timestamps in the format [yyyy-MM-dd_HH:mm:ss.SSS].
Use the timestamps to query the content of the logs, see Query Server Logs at a Specific Timestamp.
Note: Server log files are deleted after a defined period of days, see Server Setting - Web Server.
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/log/?req_type=server |
Timestamp 2023-02-20_09:49:10.055 2023-02-15_13:49:38.401 |
Query Server Logs at Specific Timestamps #
| URL | Description |
|---|---|
http(s)://[host]:[port]/log/?req_type=server×tamp=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns the server logs of the specified timestamp. |
Tip: Query timestamps that correspond to server logs, see Query all Server Logs.
To query server logs before, after or between timestamps, see Query Logs at Specific Timestamps.
Note: Server log files are deleted after a defined period of days, see Server Setting - Web Server.
Response
The web call returns the following information:
- LineCount: row number of the log entry.
- Name: name of the server.
- Timestamp: timestamp of the server log.
- State: returns the number 5 to indicate that the log is a server log.
- StateDescr: description of the state.
- LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
- Source technical name of the component that generates the log info.
- Message: content of the log.
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/?req_type=server×tamp=2023-02-20_09:49:10.228 |
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message 1,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Trying to listen on [::]:8065... 2,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Listening on [::]:8065 3,[server],2023-02-20_09:49:23.353,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51531 connected 4,[server],2023-02-20_09:49:23.372,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16240) started 5,[server],2023-02-20_09:49:24.802,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16240) exited with 0x0 - The operation completed successfully 6,[server],2023-02-20_09:49:36.257,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51533 connected 7,[server],2023-02-20_09:49:36.262,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16368) started |
Query a List of all Defined Extractions #
| URL | Description |
|---|---|
http(s)://[host]:[port]/extractions |
Returns a list of all defined extractions. |
http(s)://[host]:[port]/config/extractions/ |
Returns a list of all defined extractions with more details and in JSON format. |
Response
The web calls return the following information:
- Name: name of the extraction.
- Type: extraction type, e.g., Table, Report, ODP, etc.
- Source: name of the SAP source connection.
- Destination: name of the destination.
- LastRun: timestamp of the last execution.
- RowCount: number of the last extracted data records.
- LastChange: timestamp of the last change.
- Created: timestamp of the creation.
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/extractions |
Name,Type,Source,Destination,LastRun,RowCount,LastChange,Created MAKT,Table,ec5,csv,2022-12-15_13:30:08.921,177318,2023-02-15_13:49:38.401,2022-12-12_08:39:27.407 2LIS,ODP,ec5,AzureStorageAD,2022-08-18_10:55:00.189,59058,2023-01-20_11:26:05.641,2022-08-18_10:46:50.721 COUNTRY,Hierarchy,bw2,http-csv,2022-12-01_12:53:57.098,8,2022-12-01_12:53:53.599,2022-10-05_10:41:43.848 RLT10010,Report,ec5,csv,2023-01-12_11:11:48.975,21,2022-12-13_11:07:36.437,2022-06-30_08:24:47.755 |
https://todd.theobald.local:8165/config/extractions/ |
{
"extractions":
[
{
"name": "MAKT",
"type": "Table",
"source": "ec5",
"destination": "csv",
"latestRun": {
"started": "20221215T133008.921Z",
"duration": "PT00H00M02.850S",
"rowsCount": 177318,
"state": "FinishedNoErrors"
},
"lastChange": {
"machine": "SHERRI",
"user": "alice",
"timestamp": "20230215T134938.401Z"
},
"created": {
"machine": "SHERRI",
"user": "alice",
"timestamp": "20221212T083927.407Z"
}
}
]
}
|
Query a Specific Extraction at a Specific Timestamp #
| URL | Description |
|---|---|
http(s)://[host]:[port]/log/?req_type=extraction&name=[extraction_name]×tamp=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns detailed logs of the specified extraction at the specified timestamp. |
Tip: Query the timestamp of the last run of an extraction as described in Query a List of all Defined Extractions or query timestamps using the logs as described in Query all Logs.
Response
The web call returns the following information:
- LineCount: row number of the log entry.
- Name: name of the extraction.
- Timestamp: timestamp of the extraction.
- State: returns a number between 2 and 4 for a server extraction, see table below.
- StateDescr: description of the state, see table below.
- LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
- Source technical name of the component that generates the log info.
- Message: content of the log.
| State | StateDescr | Description |
|---|---|---|
| 2 | Running | The extraction is running. |
| 3 | FinishedNoErrors | Extraction succeeded without errors. |
| 4 | FinishedErrors | Extraction is finished with at least one error. |
For information on how to read extractions logs, see Extraction Logs.
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/log/?req_type=extraction&name=makt×tamp=2023-02-20_09:49:23.807 |
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message 1,makt,2023-02-21_11:04:33.765,3,FinishedNoErrors,Info,LiveDataExtraction,Product version 5.21.10.14 2,makt,2023-02-21_11:04:33.766,3,FinishedNoErrors,Info,LiveDataExtraction,Using Theobald.Extractors Interface 3,makt,2023-02-21_11:04:33.817,3,FinishedNoErrors,Info,LiveDataExtraction,Theobald.Extractors version 1.39.3.13 4,makt,2023-02-21_11:04:33.817,3,FinishedNoErrors,Info,LiveDataExtraction,Executing Table extraction 5,makt,2023-02-21_11:04:33.882,3,FinishedNoErrors,Info,LiveDataExtraction,Found license. 6,makt,2023-02-21_11:04:33.930,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,'Use SAPGUI' expert option is disabled 7,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,"Connecting to SAP application server, using Classic RFC SDK" 8,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,"Client '800', language 'EN'" 9,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,"User ALICE, Password has been provided" 10,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,Using plain authentication 11,makt,2023-02-21_11:04:34.291,3,FinishedNoErrors,Info,R3ConnectorServerWindows,"Connected to SAP host 'sap-erp-as05.example.com', instance 00, release 740, codepage 4103, user 'ALICE'" ... |
Query the Extraction Status #
| URL | Description |
|---|---|
http(s)://[host]:[port]/status/?name=[extraction_name]×tamp=[yyyy-MM-dd_HH:mm:ss.SSS] |
Returns the status of the specified extraction at the specified timestamp. |
Follow the steps below to create a status check routine:
- Run your extraction in asynchronous mode using the following URL schema:
http(s)://[host]:[port]/?name=[extraction_name]&wait=false
The asynchronous extraction immediately returns an HTTP-response, while the extraction is still running. For more information on how to trigger extractions, see Execute and Automate - Call via Webservice. - Copy the timestamp that is returned in the HTTP-response header and body of the asynchronous extraction.
Example: X-XU-Timestamp: 2023-01-28_09:58:47.312. - Use the extraction name and the timestamp to query the status of the extraction using the following URL schema:
http(s)://[host]:[port]/status/?name=[extraction_name]×tamp=[yyyy-MM-dd_HH:mm:ss.SSS] - The status of an extraction changes in time. Query the status of the extraction in a loop to trigger follow-up actions once the extraction is finished.
Note: Triggering an extraction in asynchronous mode and querying the extraction status is only possible with push-destinations, e.g.s databases or file destinations.
Response
The web call returns one of the following statuses:
| State | Description |
|---|---|
| Running | The extraction is running. |
| FinishedNoErrors | Extraction succeeded without errors. |
| FinishedErrors | Extraction is finished with at least one error. |
Example
| Example URL | Example Response |
|---|---|
https://todd.theobald.local:8165/status/?name=makt×tamp=2023-02-21_11:06:16.314 |
FinishedNoErrors |
https://todd.theobald.local:8165/status/?name=mara×tamp=2023-02-21_13:11:27.327 |
FinishedErrors |