Should I store XML declaration in database and return using WebApi - sql-server

Scenario:
User submit XML using WebApi and I want to store it in SQL database in XML Column and retrieve later using Ajax/WebApi
Question:
How should I store it in the database? With or without declaration/encoding? Or should I add the encoding when returning the XML?
public async IActionResult Post([FromBody]XDocument xml)
{
var entity = new MyDocument();
entity.Xml = xml.ToString(); //??
db.Documents.Add(entity);
db.SaveChanges();
return Created();
}
public async Task<IActionResult> Get(int id)
{
var entity = db.Documents.Find(id);
return Content(entity.Xml, "application/xml"); //missing xml declaration
}
My Observations:
XDocument.ToString() trims the XML declaration element:
var xml = XDocument.Load(#"<?xml version=""1.0"" encoding=""utf-8""?>
<Root><Child>Content</Child></Root>");
xml.ToString(); //<Root><Child>Content</Child></Root>
It's easy to include it, but I tought that maybe it's for a reason.
Edge browser does not display the XML when the response does not include xml declaration:
public IActionResult Get()
{
return Content("<Root><Child>Content</Child></Root>", "application/xml")
}
When the response include xml declaration, but the encoding from the declaration does not match response encoding, it also fails with "Unable to switch encodings":
public IActionResult Get()
{
return Content(#"<?xml version=""1.0"" encoding=""utf-8""?>
<Root><Child>Content</Child></Root>", "application/xml");
}
In order to make Edge browser to display the XML properly, I have to do folowing:
public IActionResult Get()
{
string xml = #"<?xml version=""1.0"" encoding=""utf-8""?>
<Root><Child>Content</Child></Root>")
var xmlDoc = XDocument.Parse(xml);
return Content(xml, "application/xml", Encoding.GetEncoding(xmlDoc.Declaration.Encoding));
}
Since database also has some encoding, it's quite unclear to me, what is actually the right way.

I've identified these rules as best practices:
You should not store XML encoding in database
declaration is optional, but if present, it must not include encoding
Your WebApi should always return XML Declaration.
encoding is optional, but if present, it must match response encoding
Explanation:
It's clear that you don't want to store encoding in the database, since the database threats XML columns on it own. However, it makes sense to store XML declaration in the database, because it can contain other information (version, standalone).
Your REST service should always return XML declaration in the response. In case it is not stored in database, you can construct it at runtime.
public async IActionResult Post([FromBody]XDocument xml)
{
var entity = new MyDocument();
var declarationWithoutEncoding = new XDeclaration(xml.Declaration?.Version, null, xml.Declaration?.Standalone);
entity.Xml = $#"{declarationWithoutEncoding}\n\r{xml.ToString()}"
db.Documents.Add(entity);
db.SaveChanges();
return Created();
}
public async Task<IActionResult> Get(int id)
{
var entity = db.Documents.Find(id);
return Content(entity.Xml, "application/xml"); //in case XML Declaration is in database
}
alternativelly
public async Task<IActionResult> Get(int id)
{
var entity = db.Documents.Find(id);
XDocument xmlDoc = XDocument.Parse(entity.xml);
var declaration = new XDeclaration(xml.Declaration?.Version ?? "1.0", null, xml.Declaration?.Standalone ? "no");
//you may change enccoding information to match response encoding, or just skip it
//declaration.Encoding = ...
string xml = $#"{declaration}\n\r{xml.ToString()}"
return Content(xml, "application/xml");
}

Related

parse XML message using SPEL

In my Spring Integration pipeline I am getting a XML payload and depending on the value of the attributes in the XML I have to generate a key and publish it to kafka.
return IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(kafkaListenerContainer))
.wireTap(ACARS_WIRE_TAP_CHNL) // Log the raw message
.enrichHeaders(h ->h.headerFunction(KafkaHeaders.MESSAGE_KEY, m -> {
StringBuilder header = new StringBuilder();
Expression expression = new SpelExpressionParser().parseExpression("payload.Body.toString()");
//Expression expression = new SpelExpressionParser().parseExpression("m.payload.Body.ACIFlight.fltNbr.toString()");
String flightNbr = expression.getValue(String.class);
header.append(flightNbr);
return header.toString();
}))
.get();
XMl is
<?xml version="1.0" encoding="UTF-8"?>
<ns0:Envelope xmlns:ns0="http://www.exmaple.com/FlightLeg">
<ns0:Header>
<ns1:eventHeader xmlns:ns1="http://www.exmaple.com/header" eventID="659" eventName="FlightLegEvent" version="1.0.0">
<ns1:eventSubType>FlightLeg</ns1:eventSubType>
</ns1:eventHeader>
</ns0:Header>
<ns0:Body>
<ns1:ACIFlight xmlns:ns1="http://ual.com/cep/aero/ACIFlight">
<flightKey>1267:07042020:UA</flightKey>
<fltNbr>1267</fltNbr>
<fltLastLegDepDt>07042020</fltLastLegDepDt>
<carrCd>UA</carrCd>
</ns1:ACIFlight>
</ns0:Body>
</ns0:Envelope>
I am trying to get the fltNbr from this xml payload using spel. Please suggest
Updated
String flight = XPathUtils.evaluate(message.getPayload(), "/*[local-name() = 'fltNbr']",XPathUtils.STRING);
String DepDate = XPathUtils.evaluate(message.getPayload(), "/*[local-name() = 'fltLastLegDepDt']",XPathUtils.STRING);
return MessageBuilder.fromMessage(message).setHeader("key", flight+DepDate).build();
You can use the XPath Header Enricher.
XPath is also available as a Spel function, but you'd be better off using the enricher in this case.
public class XPathHeaderEnricher extends HeaderEnricher {
Here's a test case...
#Test
public void convertedEvaluation() {
Map<String, XPathExpressionEvaluatingHeaderValueMessageProcessor> expressionMap =
new HashMap<String, XPathExpressionEvaluatingHeaderValueMessageProcessor>();
XPathExpressionEvaluatingHeaderValueMessageProcessor processor = new XPathExpressionEvaluatingHeaderValueMessageProcessor(
"/root/elementOne");
processor.setHeaderType(TimeZone.class);
expressionMap.put("one", processor);
String docAsString = "<root><elementOne>America/New_York</elementOne></root>";
XPathHeaderEnricher enricher = new XPathHeaderEnricher(expressionMap);
Message<?> result = enricher.transform(MessageBuilder.withPayload(docAsString).build());
MessageHeaders headers = result.getHeaders();
assertThat(headers.get("one")).as("Wrong value for element one expression")
.isEqualTo(TimeZone.getTimeZone("America/New_York"));
}

TryUpdateModel showing the curect data but db is not saving it

I'm trying to save a single item into a SQL Server tableusing TryUpdateModel. When debugging, I can see the value that needs to be updated, but the db.SaveChanges() call is not saving it.
My code:
[HttpGet]
public PartialViewResult _SubmitRev(int? id)
{
return PartialView();
}
[HttpPost]
public PartialViewResult _SubmitRev(int? id, WriterSubjectReviewVm model)
{
var loggedInUserId = User.Identity.GetUserId();
var member = db.Members.SingleOrDefault(m => m.ApplicationUserId == loggedInUserId);
var MySubjectDetails = (from c in db.subjects.Where(s => s.SubjectId == id) select c).AsNoTracking().Single();
model.rev.SubjectId = (int)id;
model.sub.SubjectId = MySubjectDetails.SubjectId;
var bad = MySubjectDetails.Bad;
model.sub.Bad = bad;
if (model.rev.GBU == "Bad")
{
int iBadRating = Convert.ToInt32(bad);
iBadRating++;
model.sub.Bad = iBadRating;
}
if (ModelState.IsValid)
{
// TryUpdateModel(model.sub, "Subject");
TryUpdateModel(model.sub);
db.SaveChanges();
return PartialView();
}
return PartialView(model);
}
Looking at your code, I would say that you aren't re-attaching your model back to the context. Let's break it down:
First, your model is coming into the method as a new object:
public PartialViewResult _SubmitRev(int? id, WriterSubjectReviewVm model)
Then you modify it a bit using data from your DB:
var MySubjectDetails = (from c in db.subjects.Where(s => s.SubjectId == id) select c).AsNoTracking().Single();
model.rev.SubjectId = (int)id;
model.sub.SubjectId = MySubjectDetails.SubjectId;
Important to note that you are pulling MySubjectDetails using .AsNoTracking(), which pulls it disconnected from the context, so this won't automatically save at all unless you re-attach it.
You then assign that disconnected entity to your model:
var bad = MySubjectDetails.Bad;
model.sub.Bad = bad;
Then you modify some more properties, then you check if the model is valid and try and save it:
if (ModelState.IsValid)
{
// TryUpdateModel(model.sub, "Subject");
TryUpdateModel(model.sub);
db.SaveChanges();
return PartialView();
}
At no point have you reconnected your model object back to the context (db), so when you call .SaveChanges(), what are you saving?
The Solution
At some stage you need to map the properties as posted to your Action (in the form of the WriterSubjectReviewVm view model) back onto a data model. Otherwise if that view model is actually a data model (and exists on your DB context in a collection somewhere) then you need to reattach it:
db.WriterSubjectReviews.Attach(model)
Or something similar - then when you call SaveChanges() it will actually save.

How to search for empty strings in a text field with Entity Framework?

I'd like to know how can I search for empty strings when I'm using a text type field with Entity Framework.
I've looked the SQL query that Entity is generating and It's using LIKE to compare because It's searching in a text type field, so when I use .Equals(""), == "", == string.Empty, .Contains(""), .Contains(string.Empty), and everything else, It's returning all results because it sql query is like '' and the == command throws exception because It uses the = command that is not valid with text type field.
When I try to use .Equals(null), .Contains(null), == null, It returns nothing, because It is generating FIELD ISNULL command.
I already tried the .Lenght == 0 but It throws an exception...
This works for me:
public class POCO
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
}
static void Main(string[] args)
{
var pocos = new List<POCO>
{
new POCO { Id = 1, Name = "John", Description = "basic" },
new POCO { Id = 2, Name = "Jane", Description = "" },
new POCO { Id = 3, Name = "Joey", Description = string.Empty }
};
pocos.Where(x => x.Description == string.Empty)
.ToList()
.ForEach(x => Console.WriteLine($"{x.Id} {x.Name} {x.Description}"));
}
However the issue MAY BE that your T4 generated object is not fully realized with data you can use, if you are using Entity Framework. EG the translation from the database is not populating objects to interrogate correctly. I would just do an operation like this to see:
using (var context = new YOURCONTEXTNAME())
{
var persons = context.YOURDATABASEOBJECT.ToList();
persons.ForEach(x => Console.WriteLine($"{x.COLUMNINQUESTION}"));
}
If you are successfully having data in it, it should be retrieved. I would not use text if possible. Use a varchar(max) nvarchar(max) xml, whatever text will be deprecated eventually and is bad form so to speak to continue using at this point.
EDIT
Okay I see, the answer is you cannot interogate the object until it is fully realized when it is text. I did a test on my local database and created a context and tested it and sure enough you cannot do a '== string.empty', '== ""', or 'String.IsNullOrEmpty()' on a text. However you can do it once the object is materialized in a realized object. EG:
// Won't work as context does not understand type
//var persons = context.tePersons.Where(x => x.Description == string.Empty).ToList();
//Works fine as transformation got the object translated to a string in .NET
var start = context.tePersons.ToList();
var persons = start.Where(x => x.Description == String.Empty).ToList();
This poses a problem obviously as you need to get ALL your data potentially before performing a predicate. Not the best means by any measure. You could do a sql object for this instead then to do a function, proc, or view to change this.

AppEngine + Datastore + Objectify: Http Request returns inconsistent responses

I am implementing AppEngine server as a backend for my Android application. I use Datastore, I query it via Objectify service and I use Endpoints that I call via URL.
I have an entity User with properties like this:
#Id
Long id;
/**
* Name or nickname of the user
*/
public String name;
#Index
public String email;
#JsonIgnore
List<Key<User>> friends;
public User()
{
devices = new ArrayList<String>();
friendsWithKey = new ArrayList<Key<User>>();
}
public static User findRecordById(Long id)
{
return ofy().load().type(User.class).id(id).now();
}
#ApiMethod(name = "friends", httpMethod = "GET", path = "users/{userId}/friends")
public JSONObject getFriends(#Named("userId") String userId)
{
User user = User.findRecordById(Long.parseLong(userId));
JSONObject friendsObject = new JSONObject();
JSONArray jsonArray = new JSONArray();
JSONObject friend;
List<User> userList = new ArrayList<User>();
if (user.friendsWithKey != null)
{
for (Key<User> id : user.friendsWithKey)
{
friend = new JSONObject();
User user1 = User.findRecordById(id.getId());
userList.add(user1);
friend.put("name", user1.name);
friend.put("email", user1.email);
friend.put("id", user1.id);user1.lastTimeOnline.getTime());
jsonArray.add(friend);
}
friendsObject.put("friends", jsonArray);
}
return friendsObject;
}
It sometimes returns only a subset of friends. It is weird and I do not get where I could go wrong. If I get the User object from the DB, it already has a wrong List of Key values. But if I look into the database via console, I can see all of the users that ahve been added as friends.
I reaally need to fix this bug. Please, help.
It is very strange because it only happens once in a while and is non-deterministic in every way.

Nullpointerexception throws when inserting entity using Auto-generated Classendpoint insert method

I am confused to using auto-generated endpoint class. I want to use generated endpoint to insert new object into datastore. But, an exception is throwing.
fooEndpoint.insertFoo(foo); // throws null pointer exception
My entity class is similar with the given example at this source: https://developers.google.com/appengine/docs/java/datastore/jpa/overview.
Here is my entity:
#Entity
public class Foo {
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Key ID;
Here is the stack trace:
java.lang.NullPointerException
at org.datanucleus.api.jpa.JPAEntityManager.find(JPAEntityManager.java:318)
at org.datanucleus.api.jpa.JPAEntityManager.find(JPAEntityManager.java:256)
at com.FooEndpoint.containsFoo(FooEndpoint.java:150)
at com.FooEndpoint.insertFoo(FooEndpoint.java:96)
On the other side, I can insert new object when I use the EntityManager persist method. Because, this does not check exist or not on the datastore.
I expect that, classEndpoint insert method should save the object and assing auto key to ID field.
Or I need to initialize the ID field.
Here is auto-generated endpoint class insertFoo method.
/**
* This inserts a new entity into App Engine datastore. If the entity already
* exists in the datastore, an exception is thrown.
* It uses HTTP POST method.
*
* #param foo the entity to be inserted.
* #return The inserted entity.
*/
public Foo insertFoo(Foo foo) {
EntityManager mgr = getEntityManager();
try {
if (containsFoo(foo)) {
throw new EntityExistsException("Object already exists");
}
mgr.persist(foo);
} finally {
mgr.close();
}
return foo;
}
Here is the containsFoo method
private boolean containsFoo(Foo foo) {
EntityManager mgr = getEntityManager();
boolean contains = true;
try {
Foo item = mgr.find(Foo.class, foo.getID()); // exception occurs here
if (item == null) {
contains = false;
}
} finally {
mgr.close();
}
return contains;
}
foo.getID() is null. Because, it is new object. I am expecting that, app engine creates a key for it. Or I need to explicitly create a key for it?
Other fields in Foo class are simple types such as String and booleans.
Thanks for your time.
I had exactly the same problem.
I will present the way I worked around it.
Original auto-generated Endpoints class relevant code:
private boolean containsFoo(Foo foo) {
EntityManager mgr = getEntityManager();
boolean contains = true;
try {
Foo item = mgr.find(Foo.class, foo.getID());
if (item == null) {
contains = false;
}
} finally {
mgr.close();
}
return contains;
}
Changed relevant code to include a null check for the entity object that is passed as an argument.
private boolean containsFoo(Foo foo) {
EntityManager mgr = getEntityManager();
boolean contains = true;
try {
// If no ID was set, the entity doesn't exist yet.
if(foo.getID() == null)
return false;
Foo item = mgr.find(Foo.class, foo.getID());
if (item == null) {
contains = false;
}
} finally {
mgr.close();
}
return contains;
}
This way it will work as supposed, although I'm confident that more experienced answers and explanations will appear.
I was having the same exact problem after using the Eclipse Plugin to autogenerate the cloud endpoints (by selecting "Google > Generate Cloud Endpoint Class").
Following your advice, I added:
if(foo.getID() == null) // replace foo with the name of your own object
return false;
The problem was solved.
How is that Google hasn't updated the autogenerated code yet as this must be a highly recurring issue?
Thanks for the solution.

Resources