I have a search service running on azure in a free tier. On this service I already have a datasource, and indexer and an index defined.
I'd like to add another datasource (and index + indexer). When I do this (using postman) I get 403 Forbidden without any other error message.
This is the POST I made to this url - https://my-search-service-name.search.windows.net/datasources?api-version=2016-09-01:
"Content-Type": "application/json",
"api-key": "API-KEY-HERE"
{
"name": "datasource-prod",
"description": "Data source for search",
"type": "azuresql",
"credentials": { "connectionString" : "Server=tcp:xxxx.database.windows.net,1433;Initial Catalog=xxxxx_prod;Persist Security Info=False;User ID=xxxxxx;Password=xxxxxx!;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;" },
"container": {"name": "DataAggregatedView"},
"dataChangeDetectionPolicy": {
"#odata.type" : "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy",
"highWaterMarkColumnName" : "ChangeIndicator"
},
"dataDeletionDetectionPolicy": {
"#odata.type" : "#Microsoft.Azure.Search.SoftDeleteColumnDeletionDetectionPolicy",
"softDeleteColumnName" : "isDeleted",
"softDeleteMarkerValue" : "0"
}
}
Using the same request, with different name and database name worked perfectly and generated the existing (first) datasource. This error (403) - not even got the error message - happens only when I try to define a second datasource.
As I can understand from documentation, free search tier allows 3 datasources. Anyone had this issue? Any help/direction is appreciate!
Thank you.
Make sure you're using the admin API key. It looks like you may be using a query key.
Related
I am getting 403 errors with sp-rest-proxy. I was originally using the “User Credentials” strategy which allowed me to GET data, but not POST it. So now I’m am trying the “Addin only permissions”. My I.T. team was able get the app registered for me. but I am still receiving the below error now even with GET.
Error Details:
{
"readyState": 4,
"responseText": "{\"error\":{\"code\":\"-2147024891, System.UnauthorizedAccessException\",\"message\":{\"lang\":\"en-US\",\"value\":\"Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))\"}}}",
"responseJSON": {
"error": {
"code": "-2147024891, System.UnauthorizedAccessException",
"message": {
"lang": "en-US",
"value": "Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))"
}
}
},
"status": 403,
"statusText": "Forbidden"
}
Things I suspect I messed up on:
I strongly think its my server/private config I have the following…
const RestProxy = require('sp-rest-proxy');
const settings = {
configPath: './config/private.json',
port: 8081,
};
const restProxy = new RestProxy(settings);
restProxy.serve();
and private (not the actual values I am using expect for "strategy" )
{
"siteUrl": "https://ORGANIZTION.sharepoint.com",
"strategy": "OnlineAddinOnly",
"clientId": "0000000-000000-000000-0000-00000000",
"clientSecret": "000000000000000000000000000000",
"realm": "00000-0000-0000-0000-000000"
}
I couldn’t find much on the “strategy” value on the sp-rest-proxy or the node-sp-auth side of the documentation. I can assume its OnlineAddinOnly but I’m not able to find the specific syntax for what possible values this attribute expects. I also noticed that the “clientSecret” is changing once I run the server, I assume this is an intentional encryption.
During the App registration phase (step 5 of this https://github.com/s-KaiNet/node-sp-auth/wiki/SharePoint%20Online%20addin%20only%20authentication) I had the IT folk set the “right” attribute in AppPermissionRequests to “Write” instead “FullControl”, I noticed that “FullControl” seems to be used in most example though I wasn’t sure if it was required. Can anyone confirm that?
[Edit: confirmed this is not the issue by setting this to FullControl]
Intention:
I am trying to build an internal data management tool that only needs to work on localhost to get manipulate and replace json files in my teams SharePoint. (just in a nice way so that non-coders can do this). The “sp-rest-proxy” library seems to be what I need to implement the REST API effectively in react.
As far as I know, SharePoint app-only access is disabled by default. You need to ask your administrator to enable it by running the following command:
set-spotenant -DisableCustomAppAuthentication $false
The answer likely in the XML AppPermissionRequests. The creator of the library was able to point me to a better example and I had noticed some differences we had a different scope value and no AllowAppOnlyPolicy adding these seems to have fixed most of the issue. I am able to confirm that I can now do GET.
I am still having issues with GetFolderByServerRelativeUrl and using the to add/replace files but I am not sure that is related and will treat it as a separate issue as it may not be related to sp-rest-proxy or node-sp-auth
the correct AppPermissionRequests XML should be this ->
and as #Michael Han_MSFT mentioned you should ensure that DisableCustomAppAuthentication is set to false
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
</AppPermissionRequests>
I am facing the below issue with Azure Data Factory using Logic App.
I am using the Azure Data Factory pipeline for migration and Logic App for sending "Success & Failure" notification to the technical team.
Now success is working fine as the message is hardcoded, but failure is not as the Logic App web activity is not able to parse data factory pipeline error.
Here is the input that is going to Logic App web activity
Input
{
"url": "https://xxxxxxxxxxxxxxxxx",
"method": "POST",
"headers": {},
"body": "{\n \"title\": \"PIPELINE RUN FAILED\",\n \"message\":\"Operation on target Migration Validation failed: Execution fail against sql server. Sql error number: 50000. Error Message: The DELETE statement conflicted with the REFERENCE constraint \"FK_cmclientapprovedproducts_cmlinkclientchannel\". The conflict occurred in database \"Core7\", table \"dbo.cmClientApprovedProducts\", column 'linkclientchannelid'.\",\n \"color\": \"Red\",\n \"dataFactoryName\": \"LFC-TO-MCP-ADF\",\n \"pipelineName\": \"LFC TO MCP MIGRATION\",\n \"pipelineRunId\": \"f4f84365-58f0-4da1-aa00-64c3a4daa9e1\",\n \"time\": \"2020-07-31T22:44:01.6477435Z\"\n}"
}
Here is the error logic app is throwing
failures
{
"errorCode": "2108",
"message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: F. Path 'message', line 3, position 202.'.\"}}",
"failureType": "UserError",
"target": "Send Failed Notification",
"details": []
}
I have tried various options, like set variable and convert by using various existing methods (string, json, replace etc), but no luck
e.g #string(activity('LOS migration').Error.Message)
Struggling almost all day this...please suggest if anyone faced a similar issue...
Below is the data flow activity
now it is working...
Pasting body content into the body text field box WITHOUT clicking on 'Add Dynamic Content' in web activity calling Logic App.
For the failure case, pass the error output use #{activity('LOS migration').error.message.
For sending email, it doesn't know if it's going to send a failure or success email. We have to adapt the body so the activity can use parameters, which we'll define later:
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#{pipeline().parameters.Subject}",
"ErrorMessage": "#{pipeline().parameters.ErrorMessage}",
"EmailTo": "#pipeline().parameters.EmailTo"
}
We can reference this variables in the body by using the following format: #pipeline().parameters.parametername. For more details, you could refer to this article.
If you want to use the direct error message of the data factory activity as an input to the logic app email expression, you could try.
"ErrorMessage": "#{string(replace(activity('activity_name').Error.Message, '"',''''))}"
Replace 'activity_name' with your failing activity name.
I would like to add some custom data to emails and to be able to filter them by using GraphAPI.
So far, I was able to create a Schema Extension and it gets returned successfully when I query https://graph.microsoft.com/v1.0/schemaExtensions/ourdomain_EmailCustomFields:
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#schemaExtensions/$entity",
"id": "ourdomain_EmailCustomFields",
"description": "Custom data for emails",
"targetTypes": [
"Message"
],
"status": "InDevelopment",
"owner": "hiding",
"properties": [
{
"name": "MailID",
"type": "String"
},
{
"name": "ProcessedAt",
"type": "DateTime"
}
]
}
Then I patched a specific message https://graph.microsoft.com/v1.0/me/mailFolders/Inbox/Messages/hidingmessageid:
PATCH Request
{"ourdomain_EmailCustomFields":{"MailID":"12","ProcessedAt":"2020-05-27T16:21:19.0204032-07:00"}}
The problem is that when I select the message, the added custom data doesn't appear by executing a GET request: https://graph.microsoft.com/v1.0/me/mailFolders/Inbox/Messages?$top=1&$select=id,subject,ourdomain_EmailCustomFields
Also, the following GET request gives me an error.
Request: https://graph.microsoft.com/v1.0/me/mailFolders/Inbox/Messages?$filter=ourdomain_EmailCustomFields/MailID eq '12'
Response:
{
"error": {
"code": "RequestBroker--ParseUri",
"message": "Could not find a property named 'e2_someguid_ourdomain_EmailCustomFields' on type 'Microsoft.OutlookServices.Message'.",
"innerError": {
"request-id": "someguid",
"date": "2020-05-29T01:04:53"
}
}
}
Do you have any ideas on how to resolve the issues?
Thank you!
I took your schema extension and copied and pasted it into my tenant, except with a random app registration I created as owner. then patched an email with your statement, and it does work correctly.
A couple of things here,
I would verify using microsoft graph explorer that everything is correct. eg, log into graph explorer with an admin account https://developer.microsoft.com/en-us/graph/graph-explorer#
first make sure the schema extensions exists
run a get request for
https://graph.microsoft.com/v1.0/schemaExtensions/DOMAIN_EmailCustomFields
It should return the schemaextension you created.
then
Run a get request for the actual message you patched not all messages that you filtered for now.
https://graph.microsoft.com/v1.0/me/mailFolders/Inbox/Messages/MESSAGEID?$select=DOMAIN_EmailCustomFields
here the response should be the email you patched and your EmailCustomField should be in the data somewhere, if it is not, that means that your patch did not work.
then you can run patch again from graph explorer
I did all this from graph explorer, easiest way to confirm.
two other things,
1) maybe the ?$top=1 in your get first message isn't the same message that you patched?
2) as per the documentation, you cannot use $filter for schema extensions with the message entity. (https://learn.microsoft.com/en-us/graph/known-issues#filtering-on-schema-extension-properties-not-supported-on-all-entity-types) So that second Get will never work.
Hopefully this helps you troubleshoot.
I have an azure cognitive search service that I am trying to add an azure blob storage data source via the api. Creating it works fine via the portal.
Here is the uri:
https://xxxxxx.search.windows.net/datasources?api-version=2019-05-06
Here are the headers:
User-Agent: Fiddler
Content-Type: application/json
api-key: XXXXXXXXXXXXXXXXXXXXXXXXXXXX
Host: XXXXXXXXXX.search.windows.net
Content-Length: 412
Here is the body:
{
"name" : "documents",
"description" : "documents data source",
"type" : "'azureblob",
"credentials" :
{ "connectionString" :
"XXXXXXXXX"
},
"container" : { "name" : "documents" }
}
When I run it, I get a 400 error code with the following message:
{"error":{"code":"","message":"Data source type ''azureblob' is not
supported"}}
I got the enum value straight from the docs here. Am I missing something?
Thanks in advance
So it was a copy/paste error from the docs:
"azureblob" <> "azureblob "
I'm trying to connect to cloud sql inside google cloud endpoint and using lightweight jdbc wrapper sql2o as data access method.
#Api(name = "questionapi", version = "v1", description = "question api")
public class QuestionService {
private static Sql2o sql2o = new Sql2o(
"jdbc:mysql://xxx.xxx.xxx.xxx:3306/xxxxx", "root",
"xxxxxxx");
#ApiMethod(name = "get", httpMethod = HttpMethod.GET)
public List<Question> get() {
String q = "select * from questions";
try (Connection conn = sql2o.open()) {
return conn.createQuery(q).executeAndFetch(Question.class);
}
}
After the app is running, I can visit localhost:8888/_ah/api/explorer to try the api. However, there is an error says:
org.sql2o.Sql2oException: Could not acquire a connection from DataSource - No suitable driver found for jdbc:mysql://xxx.xxx.xxx.xxx:3306/xxxxx
How can I solve this issue?
EDIT:
After change to maven project and I got this new error message:
503 Service Unavailable
- Show headers -
{
"error": {
"message": "java.lang.NoClassDefFoundError: Could not initialize class com.mysql.jdbc.ConnectionImpl",
"code": 503,
"errors": [
{
"domain": "global",
"reason": "backendError",
"message": "java.lang.NoClassDefFoundError: Could not initialize class com.mysql.jdbc.ConnectionImpl"
}
]
}
}
EDIT
It's a new day, I still stuck here.
What I did is I use maven to download the endpoints-skeleton-archetype project, it's a new, empty Cloud Endpoints backend API project ready for use, with required files and directories.
I immediately deploy it to app engine, and try to return a meaningful value. It worked, a simple 'hellp world' string will be returned.
Next, I tried to connect to cloud sql using jdbc. In order to do that, I followed the tutorial here
to add <use-google-connector-j>true</use-google-connector-j> into appengine-web.xml
and I try different combination of connection string
Class.forName("com.mysql.jdbc.GoogleDriver");
String url = "jdbc:google:mysql://xxxxxxxxxxxx:xxxxx?user=root";
conn = DriverManager.getConnection(url);
ResultSet rs = conn.createStatement().executeQuery("SELECT 1 + 1");
After all these, I still get this error message.
503 Service Unavailable
- Show headers -
{
"error": {
"errors": [
{
"domain": "global",
"reason": "backendError",
"message": "java.lang.NullPointerException"
}
],
"code": 503,
"message": "java.lang.NullPointerException"
}
}
This error occurs when the jdbc driver is not found in the classpath. How are you managing dependencies? Do you use maven? The error should be fixed if you add the mysql jdbc driver to your list of dependencies.
I have another comment to your code, which has nothing to do with your question. But here it comes anyway.
The codeline below has a connection leak as you never closes the connection. This will eventually deplete the connection pool and your application will hang.
return sql2o.open().createQuery(q).executeAndFetch(Question.class);
This is the correct way of doing it when using sql2o:
try (Connection con = sql2o.open()) {
return con.createQuery(q).executeAndFetch(Question.class);
}