The request to API call datastore_v3.Put() was too large without using datastore - google-app-engine

com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large.
public static List<Area> readAreas(URL url) {
List<Area> areas = new ArrayList<Area>();
try {
BufferedReader br = new BufferedReader(new FileReader(new File(url.toURI())));
String row;
while ((row = br.readLine()) != null) {
if (row.contains(SEARCHED_ROW)) {
//get the part after "c"
String coord[] = (row.split("c"));
String startCoordM = ((coord[0].trim()).split(" "))[1];
String curvesCoord= coord[1];
Area area = new Area();
area.mPoint= Point.toStartPoint(Point.readPoints(startCoordM));
area.curves = Curve.readCurves (curvesCoord);
areas.add(area);
}
}
br.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (URISyntaxException e) {
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return areas;
}
This method runs without any errors but when I log out and log in to the same page of my web application this method runs again and again without problem but then this exception is thrown. I'm using google app engine 1.8.1 with jsf2 and primefaces 3.5. This method is invoked from managed bean :
public MapMB () {
eps = EPDAO.getEPList();
populateAdvancedModel(eps);
drawPolilines();
}
void drawPolilines() {
List<Area> areas = Area.readAreas(getFacesContext().getClass().getResource("/map-inksc.svg") );
for (Area area : areas) {
List<Curve> curves = area.getCurves();
Point endPoint = area.getmPoint();
Polyline polyline = new Polyline();
polyline.setStrokeWeight(1);
polyline.setStrokeColor("#FF0000");
polyline.setStrokeOpacity(1);
for (Curve curve : curves) {
polyline.getPaths().add( new LatLng(endPoint.getY(),endPoint.getX()) );
// curve start point is the end point of previous curve (endPoint.getX(),endPoint.getY() )
double step = 0.01;
for (double t=0;t<= 1;t=t+step) {
double x = getCoordFromCurve(endPoint.getX(), endPoint.getX() + curve.getP1().getX(),endPoint.getX() + curve.getP2().getX(),endPoint.getX() + curve.getP3().getX(), t);
double y = getCoordFromCurve(endPoint.getY(), endPoint.getY() + curve.getP1().getY(),endPoint.getY() + curve.getP2().getY(),endPoint.getY() + curve.getP3().getY(), t);
polyline.getPaths().add( new LatLng(y, x) );
}
endPoint = new Point (endPoint.getX() + curve.getP3().getX(), endPoint.getY() + curve.getP3().getY());
}
advancedModel.addOverlay(polyline);
polyline = new Polyline();
}
}
When I don't read any data (don't use readAreas() above) then everything works fine. So how reading from file is connected to this error? I don't understand.
If there is some information that I didn't put here please just say. All these methods run without errors and then this exception is thrown

See the edit
Ok. So ... somehow the problem is solved. How? I'm not sure. So I had:
a.xhtml < include b.xhtml
c.xhtml < include b.xhtml
a.xhtml and c.xhtml had the same method bFilterMethod()
JSF beans:
a, b, c all ViewScoped
b had a and c as Managed Properties
a.xhtml and c.xhtml have bFilterMethod() that getsSome() data from the database and sets aProperty and cProperty(which are the same). I saw in google app engine logs that the method getsSome() runs about 20 times like infinite loop after that the exception was thrown.
Now all beans are request scoped
a.xhtml has aFilterMethod that getsSome() data
b.xhtml has bFilterMethod that getsSome() data
and a and b has c as Managed Property
Hope this helps someone but as I sad I'm not sure what is the exact error but obviously is caused by too big request from the database no matter this request contains only 3 rows (this request is invoked too many times)
EDIT
After so many years I came back to my topic accidentally. The real reason for all this is that GAE saves the session in the datastore and jsf ViewScoped beans are not removed from the session as in normal java application server. So the solution is just don't use ViewScoped beans

Related

Using Database as Alfresco ContentStore

I'm working with Alfresco 4.2 and I need to use a table in my database as document content store.
Collecting some information hither and thither over the internet, I read that I have to just implement my custom DBContentStore DBContentWriter and DBContentReader classes. Someone told me to take as reference the FileContentStore class.
I need some help to mapping the FileContentStore in order to match my new class.
For example;
The DBContentWriter has to extend AbstractContentWriter and in the API docs I read that the only methods I have to overwrite are:
getReader() to create a reader to the underlying content
getDirectWritableChannel() to write content to the repository.
What about the second method?
protected WritableByteChannel getDirectWritableChannel()
This is called by getContentOutputStream():
public OutputStream getContentOutputStream() throws ContentIOException
{
try
{
WritableByteChannel channel = getWritableChannel();
OutputStream is = new BufferedOutputStream(Channels.newOutputStream(channel));
// done
return is;
}
The main method is the putContent(InputStream is) which wants to write content into a DB table.
public final void putContent(InputStream is) throws ContentIOException
{
try
{
OutputStream os = getContentOutputStream();
copyStreams(is, os);
Where copyStreams does something like this:
public final int copyStreams(InputStream in, OutputStream out, long sizeLimit) throws IOException
{
int byteCount = 0;
IOException error = null;
long totalBytesRead = 0;
try
{
byte[] buffer = new byte[BYTE_BUFFER_SIZE];
int bytesRead = -1;
while ((bytesRead = in.read(buffer)) != -1)
{
// We are able to abort the copy immediately upon limit violation.
totalBytesRead += bytesRead;
if (sizeLimit > 0 && totalBytesRead > sizeLimit)
{
StringBuilder msg = new StringBuilder();
msg.append("Content size violation, limit = ")
.append(sizeLimit);
throw new ContentLimitViolationException(msg.toString());
}
out.write(buffer, 0, bytesRead);
byteCount += bytesRead;
}
out.flush();
}
finally
{
try
{
in.close();
}
catch (IOException e)
{
error = e;
logger.error("Failed to close output stream: " + this, e);
}
try
{
out.close();
}
catch (IOException e)
{
error = e;
logger.error("Failed to close output stream: " + this, e);
}
}
if (error != null)
{
throw error;
}
return byteCount;
}
}
The main target is to write some code in order to write and read from the DB using these methods.
When the out.flush() is called i should have to write into the BLOB field.
thanks
Without looking at the example implementation in FileContentStore it is difficult to determine everything that getDirectWritableChennel() needs to do. Needless to say actually creating a WritableByteChannel to your database should be relatively easy.
Assuming you are using the BLOB type and you are using JDBC to get at your database then you just need to set a stream for your BLOB and turn it in to a channel.
OutputStream stream = myBlob.setBinaryStream(1);
WritableByteChannel channel = Channels.newChannel(stream);
Will you need to overwrite other methods? Maybe. If you have specific issues with those feel free to raise them.

ChannelFactory method call increse memory

I have an winform application which consumes windows service, i user ChannelFactory
to connect to service, problem is when i call service method using channel the memory usage increase and after
method execute memory not go down(even after form close), i call GC.Collect but no change
channel Create class
public class Channel1
{
List<ChannelFactory> chanelList = new List<ChannelFactory>();
ISales salesObj;
public ISales Sales
{
get
{
if (salesObj == null)
{
ChannelFactory<ISales> saleschannel = new ChannelFactory<ISales>("SalesEndPoint");
chanelList.Add(saleschannel);
salesObj = saleschannel.CreateChannel();
}
return salesObj;
}
}
public void CloseAllChannels()
{
foreach (ChannelFactory chFac in chanelList)
{
chFac.Abort();
((IDisposable)chFac).Dispose();
}
salesObj = null;
}
}
base class
public class Base:Form
{
public Channel1 channelService = new Channel1();
public Channel1 CHANNEL
{
get
{
return channelService;
}
}
}
winform class
Form1:Base
private void btnView_Click(object sender, EventArgs e)
{
DataTable _dt = new DataTable();
try
{
gvAccounts.AutoGenerateColumns = false;
_dt = CHANNEL.Sales.GetDatatable();
gvAccounts.DataSource = _dt;
}
catch (Exception ex)
{
MessageBox.Show("Error Occurred while processing...\n" + ex.Message, "Warning", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
finally
{
CHANNEL.CloseAllChannels();
_dt.Dispose();
//GC.Collect();
}
}
You're on the right track in terms of using ChannelFactory<T>, but your implementation is a bit off.
ChannelFactory<T> creates a factory for generating channels of type T. This is a relatively expensive operation (as compared to just creating a channel from the existing factory), and is generally done once per life of the application (usually at start). You can then use that factory instance to create as many channels as your application needs.
Generally, once I've created the factory and cached it, when I need to make a call to the service I get a channel from the factory, make the call, and then close/abort the channel.
Using your posted code as a starting point, I would do something like this:
public class Channel1
{
ChannelFactory<ISales> salesChannel;
public ISales Sales
{
get
{
if (salesChannel == null)
{
salesChannel = new ChannelFactory<ISales>("SalesEndPoint");
}
return salesChannel.CreateChannel();
}
}
}
Note that I've replaced the salesObj with salesChannel (the factory). This will create the factory the first time it's called, and create a new channel from the factory every time.
Unless you have a particular requirement to do so, I wouldn't keep track of the different channels, especially if follow the open/do method/close approach.
In your form, it'd look something like this:
private void btnView_Click(object sender, EventArgs e)
{
DataTable _dt = new DataTable();
try
{
gvAccounts.AutoGenerateColumns = false;
ISales client = CHANNEL.Sales
_dt = client.GetDatatable();
gvAccounts.DataSource = _dt;
((ICommunicationObject)client).Close();
}
catch (Exception ex)
{
((ICommunicationObject)client).Abort();
MessageBox.Show("Error Occurred while processing...\n" + ex.Message, "Warning", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
}
The code above gets a new ISales channel from the factory in CHANNEL, executes the call, and then closes the channel. If an exception happens, the channel is aborted in the catch block.
I would avoid using Dispose() out of the box on the channels, as the implementation in the framework is flawed and will throw an error if the channel is in a faulted state. If you really want to use Dispose() and force the garbage collection, you can - but you'll have to work around the WCF dispose issue. Google will give you a number of workarounds (google WCF Using for a start).

how do i convert these c# winform codes to be compatible on c# wpf?

hi im working on a project that uses invoke and threads.. it is a simple remote desktop program with chat.. i got a sample here on the internet in c# winform, but i would like to convert it to wpf.. i have no problem in sending message to another client using the wpf program but it cannot receive ( or cannot read) the sent messages from the others.. i think it has something to do with the thread and the invoke method, i read that wpf does invoke differently and i did try the dispatcher.invoke, but it still doesnt do the trick
pls hellp
here's the code
wait = new Thread(new ThreadStart(waitForData));
wait.Start();
that snippet above is executed when a successful connection is made in tcpclient
private void waitForData()
{
try
{
NetworkStream read = tcpclnt.GetStream();
while (read.CanRead)
{
byte[] buffer = new byte[64];
read.Read(buffer, 0, buffer.Length);
s = new ASCIIEncoding().GetString(buffer);
System.Console.WriteLine("Recieved data:" + new ASCIIEncoding().GetString(buffer));
rcvMsg = new ASCIIEncoding().GetString(buffer) + "\n";
hasNewData = true;
bool f = false;
f = rcvMsg.Contains("##");
bool comand = false;
comand = rcvMsg.Contains("*+*-");
/*File receive*/
if (f)
{
string d = "##";
rcvMsg = rcvMsg.TrimStart(d.ToCharArray());
int lastLt = rcvMsg.LastIndexOf("|");
rcvMsg = rcvMsg.Substring(0, lastLt);
NetworkStream ns = tcpclnt.GetStream();
if (ns.CanWrite)
{
string dataS = "^^Y";
byte[] bf = new ASCIIEncoding().GetBytes(dataS);
ns.Write(bf, 0, bf.Length);
ns.Flush();
}
try
{
new Recieve_File().recieve_file(rcvMsg);
}
catch (Exception ec)
{
System.Console.WriteLine(ec.Message);
}
}
/*Command-shutdown/restart/logoff*/
else if (comand)
{
string com = "*+*-";
rcvMsg = rcvMsg.TrimStart(com.ToCharArray());
execute_command(rcvMsg);
}
else
{
this.Invoke(new setOutput(setOut));
Thread.Sleep(1000);
}
}
}
catch (Exception ex)
{
wait.Abort();
output.Text += "Error..... " + ex.StackTrace;
}
}
the snippet above is a code that listens if there is a message or command.. the line
this.invoke(new setoutput(setout)) is a code for appending text in the rtb
hope someone could help me thanks
You've posted a lot of code, but I'm assuming it's only the call to Control.Invoke which is causing the problem. In WPF, use Dispatcher.Invoke (or Dispatcher.BeginInvoke) instead, via the Dispatcher property on the relevant UI element.
I'd also strongly encourage you to:
Refactor your code into smaller methods
Stop catching just Exception except at the top level of any large operation (it should just be a fall-back; usually you catch specific exceptions)
Start following .NET naming conventions
Add a using directive for System so you can just write Console.WriteLine instead of System.Console.WriteLine everywhere
Use Encoding.ASCII instead of creating a new ASCIIEncoding each time you need one
Use a StreamReader to read character data from a stream, instead of reading it as binary data first and then encoding it
For either Stream or TextReader, don't ignore the return value from Read - it tells you how many bytes or characters have been read

BlackBerry Curve not enough storage exception

I have an application that downloads a video file that is roughly 6mb
I am trying to run this application on a Blackberry Curve 9360, which has 32mb of "media" storage
Sometimes this application runs and is able to download the video with no problems, however other times part way thru downloading the download process fails with an IO exception that states: "There is not enough free memory on the file system to complete this action"
after it fails in this manner I can open up the BlackBerry Desktop software and check the files section and see that the device is indeed reporting that 32/32 mb are full.
If I then restart the device with alt-shift-del and open up blackberry desktop software again the used space has shrunk back down to only 5-6 / 32mb full
Sometimes at this point I am able to run my application now and have it succeed the download, but other times it again gives me the same storage full error. The only thing I can notice that seems like it might be affecting whether or not it fails is how long the download takes total (i.e. it succeeds on wifi, and on good 3g signal and fails on poorer 3g signal, but this is anecdotal at best)
I have used this exact same application on a few different blackberry devices, including a few other Curve devices with the same storage size, and never run into this problem before.
My question is: Has anyone seen a BlackBerry curve device behave in such a way that it will report an incorrect storage space that gets fixed by a reboot?
And is there anything about this download code that could be causing this behavior?
class DownloadThread extends Thread {
public void run()
{
HttpConnection httpConn = null;
InputStream is = null;
try{
httpConn = (HttpConnection)Connector.open(videoUrl + ";interface=wifi");
is = httpConn.openInputStream();
}catch(IOException e){
try{
httpConn = (HttpConnection)Connector.open(videoUrl);
is = httpConn.openInputStream();
}catch(IOException ioe){
System.out.println("891: "+e.toString());
}
}
try{
if (!videoFconn.exists())
videoFconn.create();
else{
videoFconn.delete();
videoFconn.create();
}
OutputStream os = videoFconn.openOutputStream();
lengthOfWebFile = httpConn.getLength();
total = 0;
System.out.println("##################### length of web file = " + lengthOfWebFile + " #################");
byte data[] = new byte[256];
while ((count = is.read(data)) != -1) {
total += count;
progress = (int)(total*100/lengthOfWebFile);
if(model.getValue() < progress){
UiApplication.getUiApplication().invokeLater(new Runnable()
{
public void run()
{
EmbeddedMediaScreen.this.model.setValue(progress);
}
});
}
//write this chunk
os.write(data, 0, count);
Thread.yield();
}
os.flush();
os.close();
is.close();
httpConn.close();
lengthOfLocalFile = videoFconn.fileSize();
System.out.println("###################### Local Length = " + lengthOfLocalFile + "#####################");
if(lengthOfLocalFile == lengthOfWebFile){
amDownloading = false;
startVideo();
}else{
downloadVideo();
}
}catch(FileNotFoundException fnf){
}catch(IOException e){
//ScreenSaverActivity.errorDialog("975: "+e.toString());
System.out.println("980: "+e.toString());
//e.printStackTrace();
}catch(NullPointerException npe){
System.out.println("983: "+npe.toString());
} /*catch (InterruptedException e) {
// TODO Auto-generated catch block
System.out.println(e.getMessage());
}*/
}
public synchronized void postProgress(final int p){
UiApplication.getUiApplication().invokeLater(new Runnable()
{
public void run()
{
//Set the progress bar
EmbeddedMediaScreen.this.model.setValue(p);
}
});
}
}

VB.NET Webbrowser System.UnauthorizedAccessException in Loop

I've had this code working for at least a year and today it threw an exception that i haven't been able to figure out why its happening. Its a Forms.WebBrowser that hits a generic site first and then a secondary site.
'first site
wbr.ScriptErrorsSuppressed = False
wbr.Navigate("http://www.bing.com/?rb=0")
Do
Application.DoEvents()
Loop Until wbr.ReadyState = WebBrowserReadyState.Complete
'second site
wbr.ScriptErrorsSuppressed = True
Dim start As DateTime = DateTime.Now
Dim loopTimeout As TimeSpan = TimeSpan.FromSeconds(timeout)
wbr.Navigate("http://www.FlightAware.com")
Do
Application.DoEvents()
'loop timer
If DateTime.Now.Subtract(start) > loopTimeout Then
'stop browser
wbr.Stop()
'throw exception
Dim eExpTme As Exception = New Exception("A loop timeout occurred in the web request.")
Throw eExpTme
End If
Loop Until wbr.ReadyState = WebBrowserReadyState.Complete
The error happens on the second site access and it shows that it errors on the very last line with
System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
at System.Windows.Forms.UnsafeNativeMethods.IHTMLLocation.GetHref()
at System.Windows.Forms.WebBrowser.get_Document()
at System.Windows.Forms.WebBrowser.get_ReadyState()
I just don't get why its errorring on the second site and not the first and what exactly that error message means. I've looked at some help forums but nothing concrete that i can use to troubleshoot.
AGP
The web site has a frame on ad.doubleclick.net, by default cross-domain frame access is disabled for the internet zone, so you get a security exception.
Catch the exception and move on. There isn't much you need to care about in the frame, doubleclick is an ad service.
You can implement IInternetSecurityManager and let IE to believe ad.doubleclick.net and FlightAware.com are the same web site, but this can cause security problem if you extend the trust to arbitrary web sites.
Here is a little hack in C# which you can convert in Vb.net:
public class CrossFrameIE
{
// Returns null in case of failure.
public static IHTMLDocument2 GetDocumentFromWindow(IHTMLWindow2 htmlWindow)
{
if (htmlWindow == null)
{
return null;
}
// First try the usual way to get the document.
try
{
IHTMLDocument2 doc = htmlWindow.document;
return doc;
}
catch (COMException comEx)
{
// I think COMException won't be ever fired but just to be sure ...
if (comEx.ErrorCode != E_ACCESSDENIED)
{
return null;
}
}
catch (System.UnauthorizedAccessException)
{
}
catch
{
// Any other error.
return null;
}
// At this point the error was E_ACCESSDENIED because the frame contains a document from another domain.
// IE tries to prevent a cross frame scripting security issue.
try
{
// Convert IHTMLWindow2 to IWebBrowser2 using IServiceProvider.
IServiceProvider sp = (IServiceProvider)htmlWindow;
// Use IServiceProvider.QueryService to get IWebBrowser2 object.
Object brws = null;
sp.QueryService(ref IID_IWebBrowserApp, ref IID_IWebBrowser2, out brws);
// Get the document from IWebBrowser2.
IWebBrowser2 browser = (IWebBrowser2)(brws);
return (IHTMLDocument2)browser.Document;
}
catch
{
}
return null;
}
private const int E_ACCESSDENIED = unchecked((int)0x80070005L);
private static Guid IID_IWebBrowserApp = new Guid("0002DF05-0000-0000-C000-000000000046");
private static Guid IID_IWebBrowser2 = new Guid("D30C1661-CDAF-11D0-8A3E-00C04FC9E26E");
}
// This is the COM IServiceProvider interface, not System.IServiceProvider .Net interface!
[ComImport(), ComVisible(true), Guid("6D5140C1-7436-11CE-8034-00AA006009FA"),
InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]
public interface IServiceProvider
{
[return: MarshalAs(UnmanagedType.I4)]
[PreserveSig]
int QueryService(ref Guid guidService, ref Guid riid, [MarshalAs(UnmanagedType.Interface)] out object ppvObject);
}

Resources