Related
I Am working with flink 1.15.2, should i use Row or GenericRowData that inherit RowData for my own data type?, i mostly use streaming api.
Thanks.
Sig.
In general the DataStream API is very flexible when it comes to record types. POJO types might be the most convenient ones. Basically any Java class can be used but you need to check which TypeInformation is extracted via reflection. Sometimes it is necessary to manually overwrite it.
For Row you will always have to provide the types manually as reflection cannot do much based on class signatures.
GenericRowData should be avoided, it is rather an internal class with many caveats (strings must be StringData and array handling is not straightforward). Also GenericRowData becomes BinaryRowData after deserialization. TLDR This type is meant for the SQL engine.
The docs are actually helpful here, I was confused too.
The section at the top titled "All Known Implementing Classes" lists all the implementations. RowData and GenericRowData are described as internal data structures. If you can use a POJO, then great. But if you need something that implements RowData, take a look at BinaryRowData, BoxedWrapperRowData, ColumnarRowData, NestedRowData, or any of the implementations there that aren't listed as internal.
I'm personally using NestedRowData to map a DataStream[Row] into a DataStream[RowData] and I'm not at all sure that's a good idea :) Especially since I can't seem to add a string attribute
I have been struggling with creating Dynamic user interfaces or forms in Codenameone. InstantUI will work with PropertyBusinessObjects. However, the current documentation assumes that these will be defined a priori. I want a situation where I can go from a map to InstantUI. My real use case is to create an InstantUI from a JSONObject. IS this possible? If so how?
There is currently no support for that in the InstantUI class as the PropertyBusinessObject contains far more information than just key/value pairs e.g. it contains type information, labels, constraints etc.
So no that's not possible. However, you can look at the source of InstantUI and use that as a starting point to create your own implementation that doesn't require these things or fetches them from a different source.
"Make things as simple as possible, but no simpler."
Can we find the solution/s that fix the Python database world?
Update: A 'lustdb' prototype has been written by Alex Martelli - if you know any somewhat lightweight, high-level database libraries with multiple backends we could wrap in syntax sugar honey, please weigh in!
from someAmazingDB import *
#we imported a smart model class and db object which talk to database adapter/s
class Task (model):
title = ''
done = False #native types not a custom object we have to think about!
db.taskList = []
#or
db.taskList = expandableTypeCollection(Task) #not sure what this syntax would be
db['taskList'].append(Task(title='Beat old sql interfaces',done=False))
db.taskList.append(Task('Illustrate different syntax modes',True)) # ok maybe we should just use kwargs
#at this point it should be autosaved to a default db option
#by default we should be able to reload the console and access the default db:
>> from someAmazingDB import *
>> print 'Done tasks:'
>> for task in db.taskList:
>> if task.done:
>> print task.title
'Illustrate different syntax modes'
I'm a fan of Python, webPy and Cherry Py, and KISS in general.
We're talking automatic Python to SQL type translation or NoSQL.
We don't have to totally be SQL compatible! Just a scalable subset or ignore it!
Re:model changes, it's ok to ask the developer when they try to change it or have a set of sensible defaults.
Here is the challenge: The above code should work with very little modification or thinking required. Why must we put up with compromise when we know better?
It's 2010, we should be able to code scalable, simple databases in our sleep.
If you think this is important, please upvote!
What you request cannot be done in Python 2.whatever, for a very specific reason. You want to write:
class Task(model):
title = ''
isDone = False
In Python 2.anything, whatever model may possibly be, this cannot ever allow you to predict any "ordering" for the two fields, because the semantics of a class statement are:
execute the body, thus preparing a dict
locate the metaclass and run special methods thereof
Whatever the metaclass may be, step 1 has destroyed any predictability of the fields' order.
Therefore, your desired use of positional parameters, in the snippet:
Task('Illustrate different syntax modes', True)
cannot associate the arguments' values with the model's various fields. (Trying to guess by type association -- hoping no two fields ever have the same type -- would be even more horribly unpythonic than your expressed desire to use db.tasklist and db['tasklist'] indifferently and interchangeably).
One of the backwards-incompatible changes in Python 3 was introduced specifically to deal with situations of this ilk. In Python 3, a custom metaclass can define a __prepare__ function which runs before "step 1" in the above simplified list, and this lets it have more control about the class's body. Specifically, quoting PEP 3115...:
__prepare__ returns a dictionary-like object which is used to store
the class member definitions during evaluation of the class body.
In other words, the class body is evaluated as a function block
(just like it is now), except that the local variables dictionary
is replaced by the dictionary returned from __prepare__. This
dictionary object can be a regular dictionary or a custom mapping
type.
...
An example would be a metaclass that
uses information about the
ordering of member declarations to create a C struct. The metaclass
would provide a custom dictionary that simply keeps a record of the
order of insertions.
You don't want to "create a C struct" as in this example, but the order of fields is crucial (to allow the use of positional parameters that you want) and so the custom metaclass (obtained through base model) would have a __prepare__ classmethod returning an ordered dictionary. This removes the specific issue, but, of course, only if you're willing to switch all of your code using this "magic ORM" to Python 3. Would you be?
Once that's settled, the issue is, what database operations do you want to perform, and how. Your example, of course, does not clarify this at all. Is the taskList attribute name special, or should any other attribute assigned to the db object be "autosaved" (by name and, what other characteristic[s]?) and "autoretrieved" upon use? Are there to be ways to remove entities, alter them, locate them (otherwise than by having once been listed in the same attribute of the db object)? How does your sample code know what DB service to use and how to authenticate to it (e.g. by userid and password) if it requires authentication?
The specific tasks you list would not be hard to implement (e.g. on top of Google App Engine's storage service, which does not require authentication nor specification of "what DB service to use"). model's metaclass would introspect the class's fields and generate a GAE Model for the class, the db object would use __setattr__ to set an atexit trigger for storing the final value of an attribute (as an entity in a different kind of Model of course), and __getattr__ to fetch that attribute's info back from storage. Of course without some extra database functionality this all would be pretty useless;-).
Edit: so I did a little prototype (Python 2.6, and based on sqlite) and put it up on http://www.aleax.it/lustdb.zip -- it's a 3K zipfile including 225-lines lustdb.py (too long to post here) and two small test files roughly equivalent to the OP's originals: test0.py is...:
from lustdb import *
class Task(Model):
title = ''
done = False
db.taskList = []
db.taskList.append(Task(title='Beat old sql interfaces', done=False))
db.taskList.append(Task(title='Illustrate different syntax modes', done=True))
and test1.p1 is...:
from lustdb import *
print 'Done tasks:'
for task in db.taskList:
if task.done:
print task
Running test0.py (on a machine with a writable /tmp directory -- i.e., any Unix-y OS, or, on Windows, one on which a mkdir \tmp has been run at any previous time;-) has no output; after that, running test1.py outputs:
Done tasks:
Task(done=True, title=u'Illustrate different syntax modes')
Note that these are vastly less "crazily magical" than the OP's examples, in many ways, such as...:
1. no (expletive delete) redundancy whereby `db.taskList` is a synonym of `db['taskList']`, only the sensible former syntax (attribute-access) is supported
2. no mysterious (and totally crazy) way whereby a `done` attribute magically becomes `isDone` instead midway through the code
3. no mysterious (and utterly batty) way whereby a `print task` arbitrarily (or magically?) picks and prints just one of the attributes of the task
4. no weird gyrations and incantations to allow positional-attributes in lieu of named ones (this one the OP agreed to)
The prototype of course (as prototypes will;-) leaves a lot to be desired in many respects (clarity, documentation, unit tests, optimization, error checking and diagnosis, portability among different back-ends, and especially DB features beyond those implied in the question). The missing DB features are legion (for example, the OP's original examples give no way to identify a "primary key" for a model, or any other kinds of uniqueness constraints, so duplicates can abound; and it only gets worse from there;-). Nevertheless, for 225 lines (190 net of empty lines, comments and docstrings;-), it's not too bad in my biased opinion.
The proper way to continue playing with this project would of course be to initiate a new lustdb open source project on the hosting part of code.google.com (or any other good open source hosting site with issue tracker, wiki, code reviews support, online browsing, DVCS support, etc, etc) - I'd do it myself but I'm close to the limit in terms of number of open source projects I can initiate on code.google.com and don't want to "burn" the last one or two in this way;-).
BTW, the lustdb name for the module is a play of word with the OP's initials (first two letters each of first and last names), in the tradition of awk and friends -- I think it sounds nicely (and most other obvious names such as simpledb and dumbdb are taken;-).
I think you should try ZODB. It is object oriented database designed for storing python objects. Its API is quite close to example you have provided in your question, just take a look at tutorial.
What about using Elixir?
Forget ORM! I like vanilla SQL. The python wrappers like psycopg2 for postgreSQL do automatic type conversion, offer pretty good protection against SQL injection, and are nice and simple.
sql = "SELECT * FROM table WHERE id=%s"
data = (5,)
cursor.execute(sql, data)
The more I think on't the more the Smalltalk model of operation seems more relevant. Indeed the OP may not have reached far enough by using the term "database" to describe a thing which should have no need for naming.
A running Python interpreter has a pile of objects that live in memory. Their inter-relationships can be arbitrarily complex, but namespaces and the "tags" that objects are bound to are very flexible. And as pickle can explicitly serialize arbitrary structures for persistence, it doesn't seem that much of a reach to consider each Python interpreter living in that object space. Why should that object space evaporate with the interpreter's close? Semantically, this could be viewed as an extension of the anydbm tied dictionaries. And since most every thing in Python is dictionary-like, the mechanism is almost already there.
I think this may be the generic model that Alex Martelli was proposing above, it might be nice to say something like:
class Book:
def __init__(self, attributes):
self.attributes = attributes
def __getattr__(....): pass
$ python
>>> import book
>>> my_stuff.library = {'garp':
Book({'author': 'John Irving', 'title': 'The World According to Garp',
'isbn': '0-525-23770-4', 'location': 'kitchen table',
'bookmark': 'page 127'}),
...
}
>>> exit
[sometime next week]
$ python
>>> import my_stuff
>>> print my_stuff.library['garp'].location
'kitchen table'
# or even
>>> for book in my_stuff.library where book.location.contains('kitchen'):
print book.title
I don't know that you'd call the resultant language Python, but it seems like it is not that hard to implement and makes backing store equivalent to active store.
There is a natural tension between the inherent structure imposed - and sometimes desired - by RDBMs and the rather free-form navel-gazing put here, but NoSQLy databases are already approaching the content addressable memory model and probably better approximates how our minds keep track of things. Contrariwise, you wouldn't want to keep all the corporate purchase orders such a storage system, but perhaps you might.
How about you give an example of how "simple" you want your "dealing with database" to be, and I then tell you all the stuff that is needed for that "simplicity" to get working ?
(And of which it will still be YOU that will be required to give the information/config to the database interface engine, somewhere, somehow.)
To name but one example :
If your database management engine is some external machine with which you/your app interfaces over IP or some such, there is no way around the fact that the IP identity of where that database engine is running, will have to be provided by your app's database interface client, somewhere, somehow. Regardless of whether that gets explicitly exposed in the code or not.
I've been busy, here it is, released under LGPL:
http://github.com/lukestanley/lustdb
It uses JSON as it's backend at the moment.
This is not the same codebase Alex Martelli did.
I wanted to make the code more readable and reusable with different
backends and such.
Elsewhere I have been working on object oriented HTML elements
accessable in Python in similar ways, AND a library for making web.py
more minimalist.
I'm thinking of ways of using all 3 elements together with automatic
MVC prototype construction or smart mapping.
While old fashioned text based template web programming will be around
for a while still because of legacy systems and because it doesn't
require any particular library or implementation, I feel soon we'll
have a lot more efficent ways of building robust, prototype friendly
web apps.
Please see the mailing list for those interested.
If you like CherryPy, you might like the complementary ORMs I wrote: GeniuSQL (which follows a Table Data gateway model) and Dejavu (which is a complete Data Mapper).
There's far too much in this question and all its subcomments to address completely, but one thing I wanted to point out was that GeniuSQL and Dejavu have a very robust system for mapping native Python types to the types that your particular backend is using. There are very sane defaults, which can be overridden as needed, and even extended if you make a new backend or use types from a backend that isn't yet supported. See http://www.aminus.net/geniusql/chrome/common/doc/trunk/advanced.html#custom for more discussion on that.
In this stackoverflow question I learned that Prism/Unity is not as decoupled as I thought, e.g. if I have this class which gets menuManager injected into its constructor, then I have to make sure that this class actually exists somewhere (I thought that you could just pull the .dll that contains the class and the container would deal with it, e.g. injecting a null in its place):
public class EmployeesPresenter
{
public EmployeesPresenter(IMenuManager menuManager)
{
}
}
But I can deal with that: the application cannot run without a MenuModule (or else as was suggested I could have a NullMenuModule which does nothing but keeps the application from breaking).
However, the application I am building will have a MenuManager class in the MenuModule and every module will have to register everything it wants to have in the menu with the MenuManager. However, I want to be able to swap out MenuModules e.g. have a InfragisticsMenuModule and have a TelerikMenuModule, etc.
However, when I am in e.g. the CustomersModule, in order to use TelerikMenuModule, I need to reference it. And when I want to use InfragisticsMenuModule, I need to reference that.
So how will I be able to "hot swap" TelerikMenuModule with InfragisticsMenuModule without recompiling all my modules with new references, e.g. I want to replace this:
Application.exe
Customers.dll
TelerikMenuModule.dll
with this:
Application.exe
Customers.dll
InfragisticsMenuModule.dll
and simply be able to restart the application and it runs with the new InfragisticsMenuModule.dll and does not complain that TelerikMenuModule.dll no longer exists.
This is where interfaces come in. You need something like:
public interface IMenuSystem
{
// whatever operations need to be offered by a menu system
}
Application.exe and Customers.dll may only refer to that interface. They aren't allow to know about a specific implementation.
Then you would use configuration steps (calling Register... methods or using a config file) to specify which type will provide the implementation of MenuSystem.
For obvious reason MEF comes to mind here and is designed for stuffs like this. I haven't had a chance to use Unity, so I'm not sure if it has something built in like this (i.e. scanning a directory for an IMenuModule implementation), but MEF can do this.
Suggestion also is to put this IMenuModule in a common assembly (separate from your other assembly). I usually called this thing Something.Core.dll.
So you might have: Application.exe, Customer.dll, Application.Core.dll, and your specific MenuModule implementation.
Your specific MenuModule implementation will reference the Application.Core assembly to gain access to its IMenuModule interface and implement it there.
I used to have one class for one file. For example car.cs has the class car. But as I program more classes, I would like to add them to the same file. For example car.cs has the class car and the door class, etc.
My question is good for Java, C#, PHP or any other programming language. Should I try not having multiple classes in the same file or is it ok?
I think you should try to keep your code to 1 class per file.
I suggest this because it will be easier to find your class later. Also, it will work better with your source control system (if a file changes, then you know that a particular class has changed).
The only time I think it's correct to use more than one class per file is when you are using internal classes... but internal classes are inside another class, and thus can be left inside the same file. The inner classes roles are strongly related to the outer classes, so placing them in the same file is fine.
In Java, one public class per file is the way the language works. A group of Java files can be collected into a package.
In Python, however, files are "modules", and typically have a number of closely related classes. A Python package is a directory, just like a Java package.
This gives Python an extra level of grouping between class and package.
There is no one right answer that is language-agnostic. It varies with the language.
One class per file is a good rule, but it's appropriate to make some exceptions. For instance, if I'm working in a project where most classes have associated collection types, often I'll keep the class and its collection in the same file, e.g.:
public class Customer { /* whatever */ }
public class CustomerCollection : List<Customer> { /* whatever */ }
The best rule of thumb is to keep one class per file except when that starts to make things harder rather than easier. Since Visual Studio's Find in Files is so effective, you probably won't have to spend much time looking through the file structure anyway.
No I don't think it's an entirely bad practice. What I mean by that is in general it's best to have a separate file per class, but there are definitely good exception cases where it's better to have a bunch of classes in one file. A good example of this is a group of Exception classes, if you have a few dozen of these for a given group does it really make sense to have separate a separate file for each two liner class? I would argue not. In this case having a group of exceptions in one class is much less cumbersome and simple IMHO.
I've found that whenever I try to combine multiple types into a single file, I always end going back and separating them simply because it makes them easier to find. Whenever I combine, there is always ultimately a moment where I'm trying to figure out wtf I defined type x.
So now, my personal rule is that each individual type (except maybe for child classes, by which a mean a class inside a class, not an inherited class) gets its own file.
Since your IDE Provides you with a "Navigate to" functionality and you have some control over namespacing within your classes then the below benefits of having multiple classes within the same file are quite worth it for me.
Parent - Child Classes
In many cases i find it quite helpful to have Inherited classes within their Base Class file.
It's quite easy then to see which properties and methods your child class inherits and the file provides a faster overview of the overall functionality.
Public: Small - Helper - DTO Classes
When you need several plain and small classes for a specific functionality i find it quite redundant to have a file with all the references and includes for just a 4-8 Liner class.....
Code navigation is also easier just scrolling over one file instead of switching between 10 files...Its also easier to refactor when you have to edit just one reference instead of 10.....
Overall breaking the Iron rule of 1 class per file provides some extra freedom to organize your code.
What happens then, really depends on your IDE, Language,Team Communication and Organizing Skills.
But if you want that freedom why sacrifice it for an iron rule?
The rule I always go by is to have one main class in a file with the same name. I may or may not include helper classes in that file depending on how tightly they're coupled with the file's main class. Are the support classes standalone, or are they useful on their own? For example, if a method in a class needs a special comparison for sorting some objects, it doesn't bother me a bit to bundle the comparison functor class into the same file as the method that uses it. I wouldn't expect to use it elsewhere and it doesn't make sense for it to be on its own.
If you are working on a team, keeping classes in separate files make it easier to control the source and reduces chances of conflicts (multiple developers changing the same file at the same time). I think it makes it easier to find the code you are looking for as well.
It can be bad from the perspective of future development and maintainability. It is much easier to remember where the Car class is if you have a Car.cs class. Where would you look for the Widget class if Widget.cs does not exist? Is it a car widget? Is it an engine widget? Oh maybe it's a bagel widget.
The only time I consider file locations is when I have to create new classes. Otherwise I never navigate by file structure. I Use "go to class" or "go to definition".
I know this is somewhat of a training issue; freeing yourself from the physical file structure of projects requires practice. It's very rewarding though ;)
If it feels good to put them in the same file, be my guest. Cant do that with public classes in java though ;)
You should refrain from doing so, unless you have a good reason.
One file with several small related classes can be more readable than several files.
For example, when using 'case classes', to simulate union types, there is a strong relationship between each class.
Using the same file for multiple classes has the advantage of grouping them together visually for the reader.
In your case, a car and a door do not seem related at all, and finding the door class in the car.cs file would be unexpected, so don't.
As a rule of thumb, one class/one file is the way to go. I often keep several interface definitions in one file, though. Several classes in one file? Only if they are very closely related somehow, and very small (< 5 methods and members)
As is true so much of the time in programming, it depends greatly on the situation.
For instance, what is the cohesiveness of the classes in question? Are they tightly coupled? Are they completely orthogonal? Are they related in functionality?
It would not be out of line for a web framework to supply a general purpose widgets.whatever file containing BaseWidget, TextWidget, CharWidget, etc.
A user of the framework would not be out of line in defining a more_widgets file to contain the additional widgets they derive from the framework widgets for their specific domain space.
When the classes are orthogonal, and have nothing to do with each other, the grouping into a single file would indeed be artificial. Assume an application to manage a robotic factory that builds cars. A file called parts containing CarParts and RobotParts would be senseless... there is not likely to be much of a relation between the ordering of spare parts for maintenance and the parts that the factory manufactures. Such a joining would add no information or knowledge about the system you are designing.
Perhaps the best rule of thumb is don't constrain your choices by a rule of thumb. Rules of thumb are created for a first cut analysis, or to constrain the choices of those who are not capable of making good choices. I think most programmers would like to believe they are capable of making good decisions.
The Smalltalk answer is: you should not have files (for programming). They make versioning and navigation painful.
One class per file is simpler to maintain and much more clear for anyone else looking at your code. It is also mandatory, or very restricted in some languages.
In Java for instance, you cannot create multiple top level classes per file, they have to be in separate files where the classname and filename are the same.
(C#) Another exception (to one file per class) I'm thinking of is having List in the same file as MyClass. Where I envisage using this is in reporting. Having an extra file just for the List seems a bit excessive.