How to use filter array to filter data from an API call? - azure-logic-apps

Trying to make a logic app work but just can't seem to finish off the last step - I have had a similar logic app working in the past with these steps but for the life of me i cannot remember the final steps to get it functional.
Would appreciate any help anyone can give. I've included a screenshot + some code below.
Logic app flow Picture
I have redacted some of the code for obvious reasons. Hopefully, it is enough for you to get a general feel for what is happening.
Every step works fine apart from the filter array which errors, I've tried a few combinations of the filter array but with no luck.
{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Filter_array": {
                "inputs": {
                    "from": "#body('Parse_JSON')?['data']",
                    "where": "#contains(item(), body('Select'))"
                },
                "runAfter": {
                    "Select": [
                        "Succeeded"
                    ]
                },
                "type": "Query"
            },
            "For_each": {
                "actions": {
                    "Send_Data": {
                        "inputs": {
                            "body": "#{items('For_each')}",
                            "headers": {
                                "Log-Type": "REDACTED"
                            },
                            "host": {
                                "connection": {
                                    "name": "#parameters('$connections')['azureloganalyticsdatacollector']['connectionId']"
                                }
                            },
                            "method": "post",
                            "path": "/api/logs"
                        },
                        "runAfter": {},
                        "type": "ApiConnection"
                    }
                },
                "foreach": "#body('Filter_array')",
                "runAfter": {
                    "Filter_array": [
                        "Succeeded"
                    ]
                },
                "type": "Foreach"
            },
            "HTTP": {
                "inputs": {
                    "headers": {
                        "Authorization": "API-Token REDACTED"
                    },
                    "method": "GET",
                    "queries": {
                        "start_date": "#variables('startDate')"
                    },
                    "uri": "REDACTED"
                },
                "runAfter": {
                    "Initialize_variable": [
                        "Succeeded"
                    ]
                },
                "type": "Http"
            },
            "Initialize_variable": {
                "inputs": {
                    "variables": [
                        {
                            "name": "startDate",
                            "type": "string",
                            "value": "#{addDays(utcnow(), -5, 'yyyy-MM-dd')}"
                        }
                    ]
                },
                "runAfter": {},
                "type": "InitializeVariable"
            },
            "Parse_JSON": {
                "inputs": {
                    "content": "#body('HTTP')",
                    "schema": {
                        "properties": {
                            "body": {
                                "properties": {
                                    "data": {
                                       REDACTED      
                             
                                    }
                                },
                                "type": "object"
                            },
                            "statusCode": {
                                "type": "integer"
                            }
                        },
                        "type": "object"
                    }
                },
                "runAfter": {
                    "HTTP": [
                        "Succeeded"
                    ]
                },
                "type": "ParseJson"
            },
            "Run_query_and_list_results": {
                "inputs": {
                    "body": "REDACTED",
                    "host": {
                        "connection": {
                            "name": "#parameters('$connections')['azuremonitorlogs']['connectionId']"
                        }
                    },
                    "method": "post",
                    "path": "/queryData",
                    "queries": {
                        "resourcegroups": “REDACTED”,
                        "resourcename": "REDACTED",
                        "resourcetype": "REDACTED",
                        "subscriptions": "REDACTED",
                        "timerange": "Last 7 days"
                    }
                },
                "runAfter": {
                    "Parse_JSON": [
                        "Succeeded"
                    ]
                },
                "type": "ApiConnection"
            },
            "Select": {
                "inputs": {
                    "from": "#body('Run_query_and_list_results')?['value']",
                    "select": "#item()?['checkpoint_s']"
                },
                "runAfter": {
                    "Run_query_and_list_results": [
                        "Succeeded"
                    ]
                },
                "type": "Select"
            }
        },
        "contentVersion": "1.0.0.0",
        "outputs": {},
        "parameters": {
            "$connections": {
                "defaultValue": {},
                "type": "Object"
            }
        },
        "triggers": {
            "Recurrence": {
                "recurrence": {
                    "frequency": "Hour",
                    "interval": 1
                },
                "type": "Recurrence"
            }
        }
    },
    "parameters": {
        "$connections": {
            "value": {
REDACTED
                }
            }
        }
    }
}
Below is a screenshot of what the above code produces in the filter array
Filter Array Error
Breakdown of what happens:
I do an API call for data, I put this through parse JSON. (this part works fine - unless there is something wrong with the parse JSON definitions but I don't think there is)
I then do a lookup in log analytics and put this into a list using select (this part works fine)
Then I compare these two data sets and filter out the ones that are already present based on a specific field in the API data against the output from the select. (I believe this is the problem area)
I finally put the output from the filter into log analytics (this part works fine)
Here is what the data from the Parse JSON roughly looks like (just the layout without any real data)
[
{
DATA,
"checkpoint": "UNIQUECODE",
DATA
},
{
DATA,
"checkpoint": "UNIQUECODE",
DATA
}
]
Here is the list that come out of the select statement that needs to be compared against to remove already present records:
[
"UNIQUECODE",
"UNIQUECODE",
"UNIQUECODE"
]

I believe I figured out an elegant solution to the problem.
"Filter_array": {
"inputs": {
"from": "#body('Parse_JSON')?['data']",
"where": "#not(contains(body('Select'), item()?['Checkpoint']))"
}
I hope this helps anyone else who is using the filter array.

Related

How to get a list of Coinbase CryptoCurrency Coins

I've been trying to figure out a way to get a list of all the Coins that Coinbase has listed (not necessarily for trade) but can't figure it out, in the early days it was easy as you could just login and see the list of 4 basic coins that were supported (and could hard code those values in a program and/or script).
But now they have a list of many coins listed, some as I understand, which are not available to actually trade but are listed for educational purposes (as stated on their site when looking at such coins).
I was wondering if anyone has figured out a way to get a list those coins (all supported and simply listed) perhaps with a tag of which are actually supported for trade.
I looked at the API and the REST API (using a simple GET request over HTTPS or using cURL for testing) has the following endpoints:
curl https://api.coinbase.com/v2/currencies - This lists all the Fiat currencies.
and:
curl https://api.pro.coinbase.com/products - This lists all the supported trading pairs (which is not what I'm looking for....)
Any ideas, short of logging in and parsing the html? (which could break since the site can be reformatted etc at any time).
Any help would be greatly appreciated!
perhaps not really what you asked, but you could also use https://api.pro.coinbase.com/currencies
import requests
import json
uri = 'https://api.pro.coinbase.com/currencies'
response = requests.get(uri).json()
for i in range(len(response)):
if response[i]['details']['type'] == 'crypto':
print(response[i]['id])
This will return the coins available for trading.
I'm not sure if I this is the response that you want or not. I first used the first URL that you have listed... The response from that looked like it didn't have the available coins. I then tried the below URL instead and the response does have a lot of curriencies listed on it. You can parse it by loading with JSON and looking for the fields that you want.
Also I didn't see a language posted with your question. I'm using python3 below. If you're a Linux person you can also just use curl GET from the command line. It doesn't matter the language... you just need to make a GET request to that URL and parse the response however you see fit.
To get 1 particular field you can use a line like response['data']['rates']['BTC'] to extract '0.00029200' out of the response/JSON string.
>>> r = requests.get("https://api.coinbase.com/v2/exchange-rates")
>>> response = json.loads(r.text)
>>> pprint.pprint(response)
{'data': {'currency': 'USD',
'rates': {'AED': '3.67',
'AFN': '75.22',
'ALL': '108.84',
'AMD': '487.59',
'ANG': '1.79',
'AOA': '311.37',
'ARS': '37.32',
'AUD': '1.38',
'AWG': '1.80',
'AZN': '1.70',
'BAM': '1.71',
'BAT': '9.00418244',
'BBD': '2.00',
'BCH': '0.00879160',
'BDT': '83.80',
'BGN': '1.71',
'BHD': '0.377',
'BIF': '1824',
'BMD': '1.00',
'BND': '1.58',
'BOB': '6.90',
'BRL': '3.65',
'BSD': '1.00',
'BTC': '0.00029200',
'BTN': '71.11',
'BWP': '10.41',
'BYN': '2.15',
'BYR': '21495',
'BZD': '2.02',
'CAD': '1.31',
'CDF': '1631.00',
'CHF': '0.99',
'CLF': '0.0242',
'CLP': '656',
'CNH': '6.71',
'CNY': '6.70',
'COP': '3174.95',
'CRC': '608.98',
'CUC': '1.00',
'CVE': '96.90',
'CZK': '22.50',
'DJF': '178',
'DKK': '6.52',
'DOP': '50.44',
'DZD': '118.30',
'EEK': '14.61',
'EGP': '17.68',
'ERN': '15.00',
'ETB': '28.52',
'ETC': '0.25542784',
'ETH': '0.00944599',
'EUR': '0.87',
'FJD': '2.10',
'FKP': '0.76',
'GBP': '0.76',
'GEL': '2.66',
'GGP': '0.76',
'GHS': '4.98',
'GIP': '0.76',
'GMD': '49.52',
'GNF': '9210',
'GTQ': '7.74',
'GYD': '208.55',
'HKD': '7.85',
'HNL': '24.49',
'HRK': '6.49',
'HTG': '78.37',
'HUF': '276',
'IDR': '13940.00',
'ILS': '3.63',
'IMP': '0.76',
'INR': '70.93',
'IQD': '1190.000',
'ISK': '120',
'JEP': '0.76',
'JMD': '132.72',
'JOD': '0.710',
'JPY': '109',
'KES': '100.60',
'KGS': '68.70',
'KHR': '4015.00',
'KMF': '429',
'KRW': '1114',
'KWD': '0.303',
'KYD': '0.83',
'KZT': '380.63',
'LAK': '8559.50',
'LBP': '1511.15',
'LKR': '178.40',
'LRD': '160.75',
'LSL': '13.53',
'LTC': '0.03208728',
'LTL': '3.22',
'LVL': '0.66',
'LYD': '1.385',
'MAD': '9.53',
'MDL': '17.05',
'MGA': '3465.0',
'MKD': '53.78',
'MMK': '1519.04',
'MNT': '2453.75',
'MOP': '8.08',
'MRO': '357.0',
'MTL': '0.68',
'MUR': '34.23',
'MVR': '15.49',
'MWK': '728.47',
'MXN': '19.14',
'MYR': '4.10',
'MZN': '61.87',
'NAD': '13.53',
'NGN': '361.50',
'NIO': '32.60',
'NOK': '8.43',
'NPR': '113.78',
'NZD': '1.45',
'OMR': '0.385',
'PAB': '1.00',
'PEN': '3.33',
'PGK': '3.36',
'PHP': '52.13',
'PKR': '139.30',
'PLN': '3.73',
'PYG': '6084',
'QAR': '3.64',
'RON': '4.14',
'RSD': '103.53',
'RUB': '65.47',
'RWF': '886',
'SAR': '3.75',
'SBD': '8.06',
'SCR': '13.67',
'SEK': '9.05',
'SGD': '1.35',
'SHP': '0.76',
'SLL': '8390.00',
'SOS': '582.00',
'SRD': '7.46',
'SSP': '130.26',
'STD': '21050.60',
'SVC': '8.75',
'SZL': '13.52',
'THB': '31.23',
'TJS': '9.43',
'TMT': '3.50',
'TND': '2.968',
'TOP': '2.26',
'TRY': '5.18',
'TTD': '6.77',
'TWD': '30.72',
'TZS': '2317.00',
'UAH': '27.70',
'UGX': '3670',
'USD': '1.00',
'USDC': '1.000000',
'UYU': '32.58',
'UZS': '8380.00',
'VEF': '248487.64',
'VND': '23287',
'VUV': '111',
'WST': '2.60',
'XAF': '573',
'XAG': '0',
'XAU': '0',
'XCD': '2.70',
'XDR': '1',
'XOF': '573',
'XPD': '0',
'XPF': '104',
'XPT': '0',
'YER': '250.30',
'ZAR': '13.27',
'ZEC': '0.02056344',
'ZMK': '5253.08',
'ZMW': '11.94',
'ZRX': '4.04721481',
'ZWL': '322.36'}}}
The following code:
import requests
uri = 'https://api.pro.coinbase.com/currencies'
response = requests.get(uri).json()
for i in range(len(response)):
if response[i]['details']['type'] == 'crypto':
print(response[i]['id'])
Will provide this output:
COTI
BTC
ETH
LTC
BCH
ZEC
XTZ
XRP
XLM
EOS
ALGO
DASH
ATOM
CGLD
FIL
ADA
ICP
SOL
DOT
DOGE
OXT
KNC
MIR
REP
COMP
NMR
ACH
BAND
ZRX
BAT
LOOM
UNI
YFI
LRC
CVC
DNT
MANA
GNT
REN
LINK
BAL
ETC
USDC
RLC
DAI
WBTC
NU
AAVE
SNX
BNT
GRT
SUSHI
MLN
ANKR
CRV
STORJ
SKL
AMP
1INCH
ENJ
NKN
OGN
FORTH
GTC
TRB
CTSI
MKR
UMA
USDT
CHZ
SHIB
BOND
LPT
QNT
KEEP
CLV
MASK
MATIC
OMG
POLY
FARM
FET
PAX
RLY
PLA
RAI
IOTX
ORN
AXS
QUICK
TRIBE
UST
REQ
TRU
WLUNA
you can use
curl -X GET https://api.exchange.coinbase.com/products
refer to
https://docs.cloud.coinbase.com/exchange/reference/exchangerestapi_getproducts

xquery for loop and if condition and should only return the first value

I am trying to write an xquery which has if inside a for loop.
Below is my request
<PersonIdentifiers>
<PIR PersonID="1834040001" IDType="N/A" IDNumber="NA" PrimaryID="true" Version="2" IdentiferFormat=""/>
<PIR PersonID="111111111" IDType="WEB" IDNumber="abc" PrimaryID="false" Version="3" IdentiferFormat=""/>
<PIR PersonID="222222222" IDType="N/A" IDNumber="xyz" PrimaryID="false" Version="4" IdentiferFormat=""/>
<PIR PersonID="333333333" IDType="WEB" IDNumber="aaaaaa" PrimaryID="false" Version="5" IdentiferFormat=""/>
<PIR PersonID="444444444" IDType="WEB" IDNumber="aaaaaa" PrimaryID="false" Version="6" IdentiferFormat=""/>
</PersonIdentifiers>
What I want in the response is just 1 or the first "IDNumber" of the row where IDType=WEB
I am currently using below xquery which has for loop to go through the array and return me value if IDType=WEB. But Ia m not bale to stop after the first successful value.
How can i achieve that?
My current xquery:
<ns2:account>
<acc:persons>
<acc:person>
<acc:personID>{fn:data($PersonMaintenanceResponse/ns1:PersonIdentifiers/ns1:PIR/#IDType)}</acc:personID>
{
for $PIR at $position in $PersonMaintenanceResponse//ns1:PersonIdentifiers/ns1:PIR
return
if($PersonMaintenanceResponse//ns1:PersonIdentifiers/ns1:PIR[$position]/#IDType='WEB')
then
if ($PersonMaintenanceResponse/ns1:PersonIdentifiers/ns1:PIR[$position]/#IDNumber)
then <acc:ID>{fn:data($PersonMaintenanceResponse/ns1:PersonIdentifiers/ns1:PIR[$position]/#IDNumber)}</acc:ID>
else ()
else
()
}
</acc:person>
</acc:persons>
</ns2:account>
Response from my xquery
<ns2:account>
<acc:persons xmlns:acc="***************">
<acc:person>
<acc:ID>abc</acc:ID>
<acc:ID>aaaaaa</acc:ID>
<acc:ID>aaaaaa</acc:ID>
</acc:person>
</acc:persons>
Expected Response is, just to get the first ID , we do not need otehr ids, how can i stop after getting 1st id
<ns2:account>
<acc:persons xmlns:acc="***************">
<acc:person>
<acc:ID>abc</acc:ID>
</acc:person>
</acc:persons>
Use predicates as filters, and index position to get the first
for $n in PersonIdentifiers/PIR[#IDType ="WEB"][1]
return
data($n/#IDNumber)
Returns abc
see:
http://xqueryfiddle.liberty-development.net/nbUY4kr/1
or as XPATH 1-liner
<account>
<persons>
<person>
<ID>{data(PersonIdentifiers/PIR[#IDType ="WEB"][1]/#IDNumber)}</ID>
</person>
</persons>
</account>

Elixir- phoenix server side rendering react using std_json_io and react-stdio

I followed the tutorial from https://medium.com/#chvanikoff/phoenix-react-love-story-reph-1-c68512cfe18 and developed an application but with different versions.
elixir - 1.3.4
phoenix - 1.2.1
poison - 2.0
distillery - 0.10
std_json_io - 0.1
The application ran successfully when running locally.
Bur when created a mix release(MIX_ENV=prod mix release --env=prod --verbose) and ran rel/utopia/bin/utopia console(the otp application name was :utopia), I ran into error
Interactive Elixir (1.3.4) - press Ctrl+C to exit (type h() ENTER for help)
14:18:21.857 request_id=idqhtoim2nb3lfeguq22627a92jqoal6 [info] GET /
panic: write |1: broken pipe
goroutine 3 [running]:
runtime.panic(0x4a49e0, 0xc21001f480)
/usr/local/Cellar/go/1.2.2/libexec/src/pkg/runtime/panic.c:266 +0xb6
log.(*Logger).Panicf(0xc210020190, 0x4de060, 0x3, 0x7f0924c84e38, 0x1, ...)
/usr/local/Cellar/go/1.2.2/libexec/src/pkg/log/log.go:200 +0xbd
main.fatal_if(0x4c2680, 0xc210039ab0)
/Users/alco/extra/goworkspace/src/goon/util.go:38 +0x17e
main.inLoop2(0x7f0924e0c388, 0xc2100396f0, 0xc2100213c0, 0x7f0924e0c310, 0xc210000000, ...)
/Users/alco/extra/goworkspace/src/goon/io.go:100 +0x5ce
created by main.wrapStdin2
/Users/alco/extra/goworkspace/src/goon/io.go:25 +0x15a
goroutine 1 [chan receive]:
main.proto_2_0(0x7ffce6670101, 0x4e3e20, 0x3, 0x4de5a0, 0x1, ...)
/Users/alco/extra/goworkspace/src/goon/proto_2_0.go:58 +0x3a3
main.main()
/Users/alco/extra/goworkspace/src/goon/main.go:51 +0x3b6
14:18:21.858 request_id=idqhtoim2nb3lfeguq22627a92jqoal6 [info] Sent 500 in 1ms
14:18:21.859 [error] #PID<0.1493.0> running Utopia.Endpoint terminated
Server: 127.0.0.1:8080 (http)
Request: GET /
** (exit) an exception was raised:
** (Protocol.UndefinedError) protocol String.Chars not implemented for {#PID<0.1467.0>, :result, %Porcelain.Result{err: nil, out: {:send, #PID<0.1466.0>}, status: 2}}
(elixir) lib/string/chars.ex:3: String.Chars.impl_for!/1
(elixir) lib/string/chars.ex:17: String.Chars.to_string/1
(utopia) lib/utopia/react_io.ex:2: Utopia.ReactIO.json_call!/2
(utopia) web/controllers/page_controller.ex:12: Utopia.PageController.index/2
(utopia) web/controllers/page_controller.ex:1: Utopia.PageController.action/2
(utopia) web/controllers/page_controller.ex:1: Utopia.PageController.phoenix_controller_pipeline/2
(utopia) lib/utopia/endpoint.ex:1: Utopia.Endpoint.instrument/4
(utopia) lib/phoenix/router.ex:261: Utopia.Router.dispatch/2
goon got panicked and hence the porcelain. Someone please provide a solution.
Related issues: https://github.com/alco/porcelain/issues/13
EDIT: My page_controller.ex
defmodule Utopia.PageController do
use Utopia.Web, :controller
def index(conn, _params) do
visitors = Utopia.Tracking.Visitors.state
initial_state = %{"visitors" => visitors}
props = %{
"location" => conn.request_path,
"initial_state" => initial_state
}
result = Utopia.ReactIO.json_call!(%{
component: "./priv/static/server/js/utopia.js",
props: props
})
render(conn, "index.html", html: result["html"], props: initial_state)
end
end

Solr Polygon Search issues

We have trouble with the solr Spatial queries some polygon queries just don´t work, we don´t know why. If you have a field of type solr.SpatialRecursivePrefixTreeFieldType
<!-- fieldType to store a location in the new format of Solr Version 4 -->
<fieldType name="location_rpt" class="solr.SpatialRecursivePrefixTreeFieldType"
spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
distErrPct="0.025"
maxDistErr="0.000009"
units="degrees"/>
and make a Query on the field with the following filter-query:
fq=fieldname:"IsWithin(POLYGON((10.949260925162957 50.687958071020525, 10.963822885556231 50.68239399969996, 10.964563833856873 50.68427156144517, 10.96703711832961 50.68658303201778, 10.971062408240995 50.68545999328907, 10.973908368601569 50.68537639297307, 10.977245419297885 50.683455151899885, 10.987904529571237 50.68294282194555, 10.991325387038 50.67963515860285, 10.994120583977482 50.67953326334638, 10.99521208549133 50.6789077661364, 11.001058877152785 50.678738633565445, 11.001548604905265 50.67818764634437, 11.001856364353715 50.677835397891904, 11.00558285688531 50.67803512409588, 11.0081512617408 50.67851875352985, 11.006105355255587 50.6765931916203, 11.007343317894337 50.675840952835756, 11.00508803742539 50.67549090681522, 11.003645117340879 50.67430948629124, 11.005941540214492 50.67061936637505, 11.006219536083469 50.67014126679768, 11.001299804762878 50.67083752214686, 11.001290248726937 50.66994678133127, 11.001266698032165 50.66775142007361, 10.994310850026004 50.6686811276132, 10.99361484218618 50.66578680941357, 10.995238751284973 50.66272061868265, 10.991757060004929 50.66144878694344, 10.98564228736983 50.656057950464636, 10.972933054968484 50.65619159367484, 10.966916033500995 50.654632430958905, 10.96636369345939 50.64608677373722, 10.962270574823638 50.645743297249716, 10.960072993917557 50.646759811319946, 10.959183606001607 50.64697929266767, 10.953790500283061 50.64683862129752, 10.950513303019612 50.645987649473184, 10.94433093410472 50.64467990104971, 10.944139287321171 50.64387083905674, 10.938406478377123 50.64367676951595, 10.936564706423367 50.64344079815689, 10.935796834794198 50.64303879575035, 10.934872743703187 50.64257439164639, 10.93427419432756 50.64211775581536, 10.933160359264956 50.641024208099076, 10.930818579187104 50.638720535539385, 10.930731698801951 50.63850491325951, 10.930573248685782 50.63819957997579, 10.930514584183914 50.63797485471301, 10.930483968228721 50.63772303174961, 10.930509448204717 50.637417013430884, 10.930559495850991 50.63670600493446, 10.930609790537858 50.636021988733695, 10.93062319254464 50.635940959080145, 10.930918118427178 50.63346819840206, 10.929554613720612 50.63173063011565, 10.928233196678372 50.63073800085285, 10.927700297053727 50.630337678088054, 10.928523643504606 50.62943775333723, 10.92872309691077 50.62921974159881, 10.928265672945901 50.62878750906588, 10.927639188094208 50.628566694106404, 10.926255509527108 50.62951454513081, 10.92251019209241 50.62758567817641, 10.922638213078281 50.62684244467462, 10.920936313471174 50.62702387937161, 10.92057977557142 50.62670125843647, 10.920152680177981 50.62639688838215, 10.918545217179695 50.62538597524043, 10.917792182353438 50.62501079438114, 10.911101933460412 50.62182259063344, 10.910534063522139 50.62158167018572, 10.910220857740773 50.62134884112989, 10.909963518297891 50.62103483348087, 10.907487776980194 50.61609483529123, 10.907185632204035 50.61550204988257, 10.907056244585032 50.61525956691028, 10.906628481384871 50.61485617486339, 10.895152937829938 50.610730036478415, 10.896242368859731 50.61247186463941, 10.896470601420656 50.61270502268978, 10.89663156493272 50.61334330885494, 10.896634921258965 50.613739197264394, 10.896609690263567 50.61409919288572, 10.896528427773802 50.6145223648505, 10.896432183751319 50.61484661316548, 10.896045213866357 50.61590967155394, 10.89592075624567 50.616243013829695, 10.895851037118929 50.616360223083014, 10.89585240754103 50.616522177442015, 10.896067734815928 50.61689934415773, 10.896183689961353 50.61723186255236, 10.896260192619177 50.6179154274055, 10.896263393697025 50.61829332089865, 10.896237700111788 50.618599331654885, 10.896016371366 50.619184942393495, 10.895837465600053 50.61977040740378, 10.895659620811532 50.62048183664009, 10.89552077688743 50.62078823450612, 10.89543827592144 50.62106744627099, 10.89541440081784 50.62158939599526, 10.895417747434317 50.621985284440285, 10.895419953189784 50.62224621091522, 10.895535536566323 50.62253374263596, 10.895650816938675 50.62278528438301, 10.895907993767457 50.623090326427935, 10.89702205813367 50.62436418220272, 10.897394394709464 50.624902766276065, 10.897581866399983 50.62532501473681, 10.897627895017118 50.625747749544246, 10.897615356639628 50.62593674492591, 10.89758943824656 50.626215763590146, 10.897451201612462 50.62659414347733, 10.897157596764943 50.62700005028155, 10.896204881347344 50.62808304669058, 10.895980612498475 50.62832675371888, 10.895770334536648 50.628552416954996, 10.89550476578125 50.62893123030415, 10.895365743634875 50.629219633049715, 10.895298437344238 50.62962476083249, 10.895301175148079 50.629948669556875, 10.895818784010213 50.63130826952094, 10.896231728685844 50.63233529670317, 10.896329308521908 50.63324913379934, 10.896471808085538 50.634162815888054, 10.896387784931676 50.63484873155491, 10.895853362204946 50.63539335124816, 10.895409731359035 50.636051929955904, 10.894831087434723 50.63668239938101, 10.89384501665182 50.63694287338006, 10.89321877746772 50.63725925036571, 10.89295453697872 50.63788864146319, 10.892601884032173 50.638689738782595, 10.892158656701907 50.639405438006676, 10.89142586330462 50.64018326820631, 10.891249605392286 50.64110163233304, 10.891209792351503 50.64141668728772, 10.891144841137194 50.64210973155851, 10.891149583241901 50.64267657195861, 10.891137616301116 50.64293754635428, 10.891225677492942 50.64331515274541, 10.891298535787737 50.64356684242054, 10.891485499888212 50.643926118463014, 10.891671788482595 50.644204417006435, 10.891786196727102 50.6443479929879, 10.892544546820737 50.64534416832987, 10.893174209281039 50.6461788173941, 10.89351799460153 50.646672522785806, 10.893705215492075 50.64705878762453, 10.894882157433013 50.650689855006604, 10.89496919918023 50.65094149403119, 10.894994089923097 50.65100770424872, 10.894997994355002 50.65101034884133, 10.89526819691057 50.651193363455, 10.896169293366368 50.651475958949085, 10.897116063005601 50.65184409397104, 10.897793725703924 50.652270282727144, 10.89865162997703 50.652752981995214, 10.89955326530773 50.65309268499627, 10.900497891510984 50.65320368900172, 10.901396850557482 50.65322914045231, 10.902342714743133 50.653482964947386, 10.903109828708763 50.6538516751205, 10.903877201611301 50.65424894803099, 10.904724676108554 50.65470480867649, 10.905207763672205 50.654919062782206, 10.905577393216744 50.65510671959086, 10.90571963130609 50.65518720031583, 10.906032572458294 50.65536605678589, 10.906360295427413 50.65561684232866, 10.908203641750418 50.65761684566244, 10.908460620376147 50.65786787497885, 10.908759750304794 50.65808276386111, 10.90914317979857 50.65822537198042, 10.9096827055135 50.65841241475139, 10.910052143839932 50.65857306489363, 10.910322344422498 50.65871607046202, 10.910593099948478 50.658922057564695, 10.911461366599966 50.65946783407124, 10.913411561596815 50.66070256810259, 10.913667868428533 50.6608726087896, 10.914080443553491 50.66110507379473, 10.914449918280287 50.66126570988579, 10.91479092498766 50.661408451414985, 10.915358821001677 50.661595366186866, 10.915898410493476 50.66178237995819, 10.916565404597831 50.66196893251853, 10.918197483579679 50.66243092630801, 10.918680067924226 50.66257314454925, 10.918978548993268 50.662707029974975, 10.919547769889176 50.6630378833151, 10.9202031483812 50.66350338704778, 10.923424641120155 50.66438241716476, 10.92520056514587 50.66507772742578, 10.928643442713414 50.66691855773872, 10.931248634299681 50.668474484061434, 10.931482632159552 50.66928340878597, 10.93237915166098 50.66977493796525, 10.934586015828625 50.671098336511534, 10.935184017704177 50.671455998806955, 10.935810765428391 50.67185853851997, 10.9360388137638 50.67201963882665, 10.936252537002147 50.67216279720219, 10.93640887044287 50.6722251919134, 10.93797214347626 50.672840129973565, 10.93838469191684 50.6730455159511, 10.938683803183944 50.673233335093705, 10.938870320684071 50.6734935625849, 10.938944158876946 50.67381720153802, 10.938904808606113 50.67415026758631, 10.938906837075631 50.674366205911554, 10.93906453424018 50.67457255591528, 10.939249959577586 50.67471581619838, 10.940034490045216 50.675324682544115, 10.940291821850632 50.67558463848537, 10.940622161521706 50.676078257435236, 10.94075248591146 50.67638368467924, 10.940783437202692 50.67666249709234, 10.943222552787118 50.678524713221, 10.944035691536161 50.67915143877554, 10.944773250404303 50.679274574957105, 10.945325950398017 50.67931743817247, 10.945708292314846 50.67931596622919, 10.946768093608876 50.68189437491155, 10.94729335869263 50.68482498983269, 10.947961248322919 50.68588856337425, 10.949260925162957 50.687958071020525)))"
(even if the index is empty) solr hangs and loads forever without error logging or showing results. This happens to a lot of queries / polygons that we try to query. What are we doing wrong, or is the polygon search not yet usable in solr 4.5?
I found the answer here: http://wiki.apache.org/solr/SolrTomcat#Enabling_Longer_Query_Requests
Enabling Longer Query Requests
If you try to submit too long a GET query to Solr, then Tomcat will reject your HTTP request on the grounds that the HTTP header is too large; symptoms may include an HTTP 400 Bad Request error or (if you execute the query in a web browser) a blank browser window.
If you need to enable longer queries, you can set the maxHttpHeaderSize attribute on the HTTP Connector element in your server.xml file. The default value is 4K. (See http://tomcat.apache.org/tomcat-5.5-doc/config/http.html)
The polygons where just too long to submit to solr via GET-request. We changed the Tomcat settings now everything works fine.

DBLib error driving me crazy

I am working on a perl script that has multiple database calls and I continuously get the following error:
DB-Library error:
Attempt to initiate a new SQL Server operation with results pending.
In my code I have the following:
The first call to the database performs a number of insert statements built from a hash:
while (my ($key, $value) = each(%holidays)) {
system("log4k", "DEBUG", "$0", "Staging holiday info data for: $cal_name: $key");
$sql = "INSERT INTO stg_holiday_data (hol_mnemonic, hol_date, hol_comment, dml_type) VALUES (\"$cal_name\", $key, \"$value\", \"N\")";
system("log4k", "TRACE", "$0", "SQL being executed: $sql");
if ($test == 0) {
$dbh->dbcmd($sql);
($dbh->dbsqlexec() && $dbh->dbresults)
or &fatalError("Database error in $sql", "DB_ERROR");
while($dbh->dbresults != NO_MORE_RESULTS) {
while(my #dat = $dbh->dbnextrow){}
}
}
}
immediately after that finishes I close off the connection, and cancel it to make sure that there are no results left to be processed by issuing:
$dbh->dbcancel();
$dbh->dbclose();
From there I call a separate subroutine to execute a stored procedure which will produce three lines of output signifying row numbers:
subroutine call
&run_sproc() if ($test == 0);
subroutine:
sub run_sproc() {
system("log4k", "DEBUG", "$0", "Loading staged holiday data");
my $sql1 = "upd_holiday_data";
system("log4k", "TRACE", "$0", "SQL being executed: $sql1");
my($dbh2) = new Sybase::DBlib $ENV{DATABASE_USER}, $ENV{DATABASE_PASSWORD}, $ENV{DATABASE_SERVER}, "GME_calendar_sync";
&fatalError("Failed to login imagine database", "DB_LOGIN_ERROR") unless ($dbh2);
$dbh2->dbcmd($sql1);
($dbh2->dbsqlexec() && $dbh2->dbresults )
or &fatalError ("Database error in $sql", "DB_ERROR");
while ($dbh2->dbresults != NO_MORE_RESULTS) {
while (my #d = $dbh2->dbnextrow) {
system("log4k", "TRACE", "$0", "Next row being inserted #d");
}
}
$dbh2->dbclose();
}
I do have a third SQL block that comes after the stored procedure that works fine with or without this subroutine.
What is happening is I am receiving the error mentioned above right before the results from the stored procedure print. I have tried everything I can imagine to make sure that all the results are being processed. A sample of the log output is below:
[Tuesday, 23 October 2012 13:30:02 BST] [DEBUG] gme_process_calendars.pl: Staging holiday info data for: CA: 20251226
[Tuesday, 23 October 2012 13:30:03 BST] [TRACE] gme_process_calendars.pl: SQL being executed: INSERT INTO stg_holiday_data (hol_mnemonic, hol_date, hol_comment, dml_type) VALUES ("CA", 20251226, "upload", "N")
[Tuesday, 23 October 2012 13:30:03 BST] [DEBUG] gme_process_calendars.pl: Staging holiday info data for: CA: 20220103
[Tuesday, 23 October 2012 13:30:03 BST] [TRACE] gme_process_calendars.pl: SQL being executed: INSERT INTO stg_holiday_data (hol_mnemonic, hol_date, hol_comment, dml_type) VALUES ("CA", 20220103, "upload", "N")
[Tuesday, 23 October 2012 13:30:03 BST] [DEBUG] gme_process_calendars.pl: Loading staged holiday data
[Tuesday, 23 October 2012 13:30:03 BST] [TRACE] gme_process_calendars.pl: SQL being executed: upd_holiday_data
DB-Library error:
Attempt to initiate a new SQL Server operation with results pending.
[Tuesday, 23 October 2012 13:30:03 BST] [TRACE] gme_process_calendars.pl: Next row being inserted 310107230
Any help with this would be greatly appreciated as I have already tried everything I can find doing an internet search and reading the documentation.
Thanks
I think your problem is that your run_proc method tries to execute this SQL:
my $sql1 = "upd_holiday_data";
which I doubt is a valid SQL command
Solution Found:
In the run_sproc sub routine, the check:
($dbh2->dbsqlexec() && $dbh2->dbresults )
or &fatalError ("Database error in $sql", "DB_ERROR");
the second condition $dbh2->dbresults is what is causing this error condition, once this was removed the error did not come up anymore.

Resources