Composer: virtual packages in provide section - package

Given that Composer can work with virtual packages in provide section of composer.json file, how are the versions managed of those virtual packages if no one holds the responsibility?
I mean if anyone can create "evil" package stating it provide-s specific virtual package (for which there is no repo anywhere), than they can specify whatever version they like. This could perhaps collide with other "good" packages, right?

From my experience, this "virtual package" feature hasn't been widely used, and it definitely has its drawbacks due to the way it is currently implemented.
If you take a look at this search result on Packagist, you'll see the three top packages being psr/log (the real interface package), psr/log (a virtual package used by two other real packages, but in the wrong way) and psr/log-implementation (used by plenty of packages, including monolog/monolog).
This example illustrates that people will misunderstand this feature and do the wrong thing. You cannot provide psr/log because that is a real package that has a real interface definition. It makes even less sense to require psr/log and at the same time also provide it.
You also correctly spotted that there is no central entity that decides which versions of a virtual package should exist, let alone which virtual package names should exist. Which isn't that much of a problem because deciding on the names of real packages is done the same way: One developer thinks of a name, and that's all. It's unlikely that this procedure creates involuntary conflicts because usually the Github account name is used as the vendor name, and Github has already made them unique. Malicious conflicts don't really exist in the real world, as Jordi has pointed out in his blog, due to the general structure Composer uses to name packages.
Back to the virtual package feature: There are two blog postings discussing it.
The first explains using this feature with the example of the virtual psr/log-implementation package mentioned above. I won't replicate that tutorial-like text here.
The second (linked and replied to at the end of the first) discusses what's bad about the whole approach of virtual packages.
Some of the points:
1) Strictly speaking (as in, would the code compile), the code from the library itself doesn't need a package that provides psr/log-implementation. It just needs the LoggerInterface (which happens to be in the psr/log package).
4) psr/log-implementation, or any virtual package for that matter, is very problematic as a package. There is no definition of this virtual package. [...]
5) Some day, someone may decide to introduce another virtual package, called the-real-psr/log-implementation (they can easily do that, right?). Such packages may be completely exchangeable with existing psr/log-implementation packages, but in order to be useful, every existing PSR-3 compliant logger package needs to mention that virtual package too in their provide section. [...]
With all these problems and uncertainties existing for good packages, it is no surprise that they do not really use this feature very much.
However, you cannot abuse it for bad things in the way you outline it in your question. Yes, any package can declare that it provides any other package. But just like it happened with psr/log, having a package declaring that it provides another package will not magically make everyone download that package. The way it works is a package declares that it provides another package, and by requiring this package, the virtual package also gets included and will fulfill any dependencies of other packages onto the virtual package.
But not requiring the package providing stuff will leave everything in it out of the equation.
In order to include bad software someone has to require it. This is best done as an indirect dependency of an innocent looking library, and requires the help of an unsuspecting developer that actively pulls this code without properly reviewing it.
Which probably is my central point for everything: If you pull in someones code into your own project, make sure you understand what that code does by reviewing it (which isn't only about malicious things, but also basic code quality, because some day you may be forced to debug a problem), or make sure you can trust that source enough to not do bad things to you. However, your own code base is not affected by packages you do not require (the last bug with such an effect was handling replace information, but I don't find that issue right now).

Related

Where or How can I find a Complete List of Available Services for DNN and 2sxc

I've been trying to both convert old code and write new code using GetScopedService().
However, I keep discovering ones I didn't know about.
Is there an easy way to find the complete list of services available for 2sxc? For DNN? And maybe even RazorBlade?
If they are not documented somewhere, is there a page of code in the public repositories that I could bookmark where it would be easy to see (and compile) a list of them?
Your best place to start is https://r.2sxc.org/services (which goes to https://docs.2sxc.org/api/dot-net/ToSic.Sxc.Services.html)
This is where we keep all the current services published. Other services are to be seen as exotic / rare use.
Razor-Blade is still mostly non-service, but we plan to fix that.
We're just about to release ServiceKits as a feature, which would make things even more intuitive. For example, ServiceKit14 has all commonly used services on it, and also IScrub from Razor Blade.

React Navigation and flow-typed

I just started using flow-typed definitions for my popular libraries in a React Native app such as React Navigation, but I find it quite hard to find the documentation on types and how to use them. I'm still getting errors in my IDE and I feel like Flow is more wasting my time than adding value to my developer experience because I've to lookup for the types all the time (and sometimes don't even find an answer). Any advice about that ?
A complex web application using many npm modules is very rarely going to be strongly typed throughout. The goal of strong typing in JS is largely to have as much typing as is feasible or even reasonable. Modules which do not have libdefs will come in as any and that's okay. Obviously it would be great if everything you pulled in had full types, but just given the way that progress is made this is practically impossible. Add to this the fact that the simple act of upgrading flow will often introduce more caught errors to your codebase, and you end up having to accept that typing is a progressive process, it shouldn't really be a blocking one.
Now that that's out of the way, you seem to have a number of different sub-questions:
I've to lookup for the types all the time
Not entirely sure what you mean by this, but you might saying it's hard to find types for the package you're using. Make sure you're familiar with how the flow-typed CLI tool works (npx flow-typed), it will help you with searching for and installing compatible libdefs. If you don't find anything for a module in flow-typed then poke around the source github repo and make sure flow types aren't shipped with the module itself. If you come across a package with a .d.ts (TypeScript) file, try converting it to a libdef with flowgen. If nothing proves fruitful, you should probably just forego types and carry on.
In this case, I would actually start my own libdef (npx flow-typed create-stub <package name>) and fill in some basic types as I went. You can start really simple, I have a libdef currently for react-select that only checks one prop of the component, the options prop (I don't remember why I have this, however :P). Again, progressive typing is the goal. Checking that one prop is actually really nice compared to checking none.
I find it quite hard to find the documentation on types and how to use them
There's generally not real documentation for libdefs in flow-typed unless it's written by the package author somewhere. I usually read the libdefs themselves, but if you find usage confusing I would recommend looking at the tests associated with the libdef. You can also dig through any relevant issues or PRs to find usage examples.
sometimes don't even find an answer
Add a $FlowFixMe and come back to it later if it slows you down too much. All of these things will become much more manageable as you become more accustomed to flow and strong typing in general, and both flow and libdefs are constantly improving.
I'm still getting errors in my IDE
If you can't fix them, add a $FlowFixMe and come back later. Flow actually has a tool included in its source code that has a utility for adding $FlowFixMe for every error, but as it's not currently shipped to npm you have to clone the source to use it.

Downsides of using Shade plugin relocation feature

To resolve jar conflicts in my application, I use the shaded plugin's relocation feature. It works for me, but i feel it is a hack. I would like to understand the downsides of using the relocation feature if any.
The main issue is that it can fail in cases where the package name is directly defined (not just imported). For example, if you are using reflection and instantiating the a class by its name (including the package name) it will generate the wrong one. Similar problems can occur when the the package is defined in a manifest (there is a transformer for that). See the plugin info for more information.
Another place where this could be an issue is with a third party dependency which uses the same dependency. Consider for example package A which is provided. If package A depends on the relocated package it will in the run time use the provided instance instead of the relocated one. This can lead to unforeseen effects.
An additional issue is that in some cases, the package may contain some initialized/static information (e.g. it downloads some info once or has some big static table). In these cases it is important to understand that there are now TWO completely separate instances of the package.

Software versioning in large-scale systems

My company is developing a system for 10 years. This system has 15 subsystems that are almost independent (they may use same libraries or packages or DBs), and these subsystems are building locally in separate teams, also a main simple system is developed to read subsystem configs and build a page with menu and submenu from configs(with shortcuts). Our company's product is the exe of this main system.
Unfortunately, we do not use a standard version numbers in our company. Now, we decided to force a standard in company, and I found Semantic Versioning a satisfactory standard, but I have some questions in our case:
How would the changes on subsystem version increase the main system version? Often, after even Major changes in a subsystem, the code of main system remains unaffected. I think changes in the main system should shape the version number of that system, but in this case it does not make sense. Is there any solution for versioning of large applications that consist of multiple subsystems?
You gave a link to MAJOR.MINOR.PATCH, and increment the
MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner,
PATCH version when you make backwards-compatible bug fixes
Any change in a subsystem might make the main system incompatable: Too much to keep track of.
Each subsystem should use their own version. In the main system you should have an overview of the dependencies and the own versioning.
You can manage this in different ways. One way would be using Maven and pom.xml files. Another way would be using a config file with the different versions.
Each team can develop their own code independent and assign Semantic versions.
Maybe some day you will put the shared libraries, packages and DBs in a repository and all teams can refer to them with their own pom.xml.
You are flexible this way. You might even want to make a second main system (for top clients or for free accounts?) and can change the pom.xml including/excluding subsystems. The second main system will have its own versioning.
From the SemVer FAQ:
What should I do if I update my own dependencies without changing the public API?
That would be considered compatible since it does not affect the public API. Software that explicitly depends on the same dependencies as your package should have their own dependency specifications and the author will notice any conflicts. Determining whether the change is a patch level or minor level modification depends on whether you updated your dependencies in order to fix a bug or introduce new functionality. I would usually expect additional code for the latter instance, in which case it's obviously a minor level increment.
I would apply this to your scenario as follows: If your main application takes upgrades of its dependent subsystems without having to make major changes in itself, it should increase either the minor or the patch version. You shouldn't need to increment the major version, as its own public API is still considered compatible.
If the main application makes use of new features added to the subsystems, it should be a minor version increase.
Otherwise, it should be a patch version increase.
I think you're on the right track for this - literally treat everything with a SemVer version as it's own thing.
If you need to rebuild an updated linked dependency then at the least it'd be a patch version update - if they're simply soft links (ie, someone can update the subsystem without rebuilding or needing fixes) then it's purely a case of wanting a version number of when it was changed.
Keeping to the SemVer idea, if things cause incompatible changes then pulling the highest change out could make sense. Ie - updating 5 subsystems at the same time, 4 are patch updates, and one a minor - you'd update the minor version of the main system only (and put them all into that single change). Same with major changes etc.
The other similar idea is to drop the changes slightly if the main build is purely an interface on top - any minor or patch changes would only cause a minor on it, and any major subsystem changes would cause a minor - that way you can keep the major for breaking front end changes etc.
Something to bear in mind is that you don't have to follow SemVer perfectly - consistency is the key for something like this - so you could update the patch level for when subsystem changes are merged in, and never reset it, while updating the minor and major versions only for the main system. (There are a couple of open source projects that work this way, I just can't remember which ones off the top of my head).
Not sure if it's any use, but you could look at some of the nodejs packages around and how they update their versions with differing dependency versions - they all include a list of what they use in a package.json file with the version listed in a certain format (ie, exactly x.y.z or >x.y.z etc - https://docs.npmjs.com/files/package.json#dependencies)

Version Control for a total newbie

I'm totally new to the world of programming and understand very little in terms of jargon and typical methodology.
A while ago I was writing some code, but accidentally deleted some good code while I was deleting bad code. From then on I started creating versions of my files, I would name each file with the date and a version number.
However, this is a pain in the ass, having to give an unique name to each file and then going to my core file and changing the reference to the name of the new file.
And then, just the other day I accidentally over wrote something important even with this method, probably because of a typo in naming.
Needless to say, this method sucks.
I'm looking for suggestions on better practices, better tools. I've been looking at version control, but a lot of them, git svn look really complicated. The idea is to speed up the whole versioning process, not make it harder by having to do command line.
Right now I'm hoping that there's a tool that would save an unique version of the file every time I hit ctrl-s, and give me one button to create a finalized version.
Of course if there are suggestions for totally different ways of doing things, that would be more awesome.
Thanks everyone.
There are two approaches to this problem:
Versioning on demand. This is the model used by subversion, CVS, etc., etc. When you have made a 'significant' change, you decide to tell the system "keep this version".
Automatic versioning. This is the model used by some old VAXen, Eclipse, IDEA, every wiki ever, and a few writer's tools. Every time you save, a new version is implicitly created. At some remove, old versions may be culled (e.g., only one version is kept from work performed a week ago, rather than every save).
It sounds like you would prefer #2, because it is "fool-proof" -- you never have to go, "oops, I should have 'checked in' / 'kept' my work before making this change." You can always roll back. One downside is that you have to manually step through the old versions to find something, because unlike with #1 you generally are not giving a description of each change.
Another downside is that for large files, or ones that are not easily diff'd/patched (i.e. binary files), you will start burning through disk space pretty fast..
As an aside, it sounds like you don't need 90% of the features in a standard SCM system -- branching, labeling, etc. -- but you might find uses for them eventually. So learning one may be a win in the long run. You can do this with svn, etc. but it will take some customizing. If you use a scriptable editor (emacs, vi, TextMate, whatever) you could redefine the "Save" command as "Save and make a new version".
Subversion is more or less the gold standard.
I'd suggest (especially for a newbie) that you check out BeanStalk (www.Beanstalkapp.com) to run your subversion server and TortoiseSVN for your client.
Good luck!
Whatever you do, if someone mentions Visual SourceSafe -- run as fast as you can. VSS was created by Satan himself and handed down to torment developers the world over.
I think you're in a position where you have to get a little bit out of your comfort zone and take some time to learn git. It's pretty easy to learn and use.
Believe me, it's really worth it. Time spent learning git is time well spent.
If you are not working in a team, you could use something like Eclipse's local history feature. It stores versions of your files locally, and you can revert to previous versions whenever you feel like it. More details here: http://help.eclipse.org/ganymede/index.jsp (Search for "local history"). I am pretty sure other IDEs have such a feature too.
If you are collaborating with others on your code, there probably is no way around learning one of the standard tools like SVN, CVS or git. For most of them, there are plugins for many IDEs available, so you don't have to use the command line.
I currently use Subversion, but my source control experience is limited.
I would however suggest reading the tutorial by Eric Sink.
http://www.ericsink.com/scm/source_control.html
Its best to learn how to use an existing 'industry standard' versioning tool like Subversion. Even if you're new to programming and version control, SVN isn't that hard to learn and will serve you well. I personally use and recommend VisualSVN Server and TortoiseSVN for Windows. Both are free and quite simple to use.
For a system that creates a revision on every save, perhaps you should look into a Versioning File System.
I think TortoiseSVN would be a good Subversion client for you to try if you're in Windows. It won't do what you're looking for with every-time-I-save-I-get-a-new-version--you'll have to manually "commit" versions to the repository. When you do a commit, that creates a new version, essentially saving your progress at that point. TortoiseSVN is pretty user-friendly, and it's a GUI, so you won't be working at the command line. You'll be able to do things like right-click a file in Windows Explorer and choose Commit to save your progress. Plus, TortoiseSVN is free and open source.
Subversion is not really complicated. If you are using Windows, TortiseSVN will help a lot, if you are using Eclipse, subclipse plug-in is awesome. (You probably should be using eclipse regardless :) )
Some of the others are a bit complicated, but you just have to know the pattern with eclipse. Maybe you could "Try it out" with an open source project or some existing subversion server.
The cycle would be:
First you "Check out" a repository. This fills up your specified directory with the contents from the repository.
If you are doing it from the command line--it's "svn co"--there is enough help there to figure out the rest.
Second you edit your files. You don't have to lock them or anything.
if you add a new file, you use "svn add filename" as soon as you add it. This won't actually change the repository until you commit your changes.
When a group of edits are done, you check them in with "svn ci" (also svn commit works).
This one has a SLIGHT twist that you'll always forget--every commit needs a comment. You don't have to specify the files you are committing or anything, but you do need to be in the top level of your project (it will commit everything below your directory.
So the procedure here is, go to the "root" of your project tree and type:
svn ci -m "comment"
piece of cake.
Finally, IF someone else is checking stuff in things get SLIGHTLY stranger. before you commit, you should "update" and get their changes. "svn up" is all it takes, but it may warn you that there were merges. This only happens when both of you edited the same file, and 90% of the time, the merges will go okay. the rest of the time, it will put little markers in your file telling you what you changed and what they changed. The "up" command will tell you which files it did this to. Go look at them and clean the file up before you check the file in.
Always test between "svn up" and "svn ci", you never know if their crappy changes busted your pristine code.
That's really it. It's so easy from the CLI, that the graphics environments are hardly worth it (but subclipse is really nice if you are in eclipse anyway because it will visually show you modified files that need to be checked in).
If you ever forget, svn's command line help is extremely terse and useful, tells you JUST what you need to know, and has help on all the sub-commands and options.
If you're looking for an easy-to-set-up version control system for Windows, I highly recommend TortoiseHg, an easy-to-use Mercurial frontend for Windows. You don't have to worry about setting up and keeping track of a repository separate from your files, but you always can do so if you'd like to. Mercurial is a great tool because it can grow with your needs. It has all the usual features like easy merging, etc. and is quite a bit easier to wrap your head around than Git in my experience.
I think Git is really easy to use especially when you use GitHub. They also provide lots of good guides to get up and running.
http://github.com
http://github.com/guides/home
I've used Git, SVN, CVS, and Perforce. On both Windows and Unix environments.
My vote is definitely for SVN, as it's ease of use, and flexibility. I prever to use command-line now, but at one time I was using TortoiseSVN for Windows, which we were able to get non-technical people to use without a hitch.
Use SVN.
You're definitely on the right track with recognizing the need for version control, but sound unsure what that might mean to you and your work. Once you learn the concepts behind version control systems, you will really come to appreciate them.
The concepts are simple: a source code control system is a piece of software designed to help you store and manage your code. How you get code into and out of it differ based on which system you choose: one paradigm is that you deliberately "check out" a file, make your changes to it, test it and make sure it's good, then check it back in. Another is that you simply save every change you make because disk space is dirt cheap, much cheaper than your time and effort spent to create the source in the first place.
Another important concept is the "baseline" or "label". When your product is in a ready-to-ship state, you tell the source code control system to create a "label" and tag every current item your entire source code base with that label. That way, when someone reports a bug in version 4.1 you can go to your system, request all the files with the "Version 4.1" label and get exactly the source code they're having a problem with.
Having a source control tool integrated with your development environment makes the whole process much easier than having to mess with command lines. (Don't discount command line because of their complexity, they deliver elegant control to an experienced user, and you eventually will become an experienced user.) But for now, I'd recommend a source code tool that can automate the process as much as possible.
Some things to consider: are you now, or are you planning to share the development with another developer? That might make a difference on how you want to set up a server. If you're developing alone on your own box, you can set it all up locally, but that's probably not the best approach for a team. (If you're unsure, git is very flexible in that arena.) Are you going to be storing large multimedia files, or just source code? Some source code systems are designed to efficiently store only text files, and will not handle movies, sounds or image files very well.
Something else to know is that most newer source control systems require some kind of "daemon" program running on the server (Subversion, git, Perforce, Microsoft Team Foundation Server) while the older, simpler systems just use the file system directly (Visual Source Safe, cvs) and don't require a server program.
If you don't want to learn much and your demands are low, the simpler solutions should suffice. Microsoft's Visual Source Safe used to come with their visual studio products, and was a very simple to use tool. It's not very robust, it's Microsoft-only, and it can't handle large files well, but it's very, very easy to set up and use. If you don't want to spend money, Subversion and git are two stellar open source solutions, and there is a lot of documentation for both on the web.
If you like to spend money, Perforce is considered an excellent choice for professional development teams (and I believe they have a free single-developer version.) If you really like to spend lots of money and want to make Bill Gates happy, Microsoft's Team Foundation Server is a complete software development lifecycle manager, is extremely easy to use in the Windows environment, and very powerful; but you'd probably want to devote an entire Windows server (plus SQL Server) instance to host it, and it will cost you several thousand dollars just on licenses. Unfortunately it is not the right tool for a one-man shop, or if you have no Windows admin experience.
If you have the budget or the connections, bringing in an experienced software engineer to help you get things started might be the quickest path to success. Otherwise, you'll have to do some more research to learn which systems best fit your situation.
Whatever VCS you use, if you choose versioning on demand instead of automatic versioning (to borrow terms from Alex's post), you will have to go through some ceremony to:
-create,
-rename,
-move,
-copy, or
-delete
a file that is under source control.
When you create a new file, you have to Add it to source control before you Commit your changes to the repository.
When you rename, move, copy, or delete a file under source control, do so with your VCS client. In TortoiseSVN and TortoiseGit, the move and copy operations are done with a right-click-and-drag, whereas the rename and delete operations are available via a right-click.
As you can imagine, changing things like the name of a project can be quite the hassle, hence the case for automatic versioning.
Ordinary file edits and any changes to files not under source control, do not require you to tell your VCS client about them.
Finally, for one-man projects, I prefer git over SVN because SVN requires at least 2 copies of everything: a repository (the "master" copy of the files and history) and a working copy (the copy you do your work on). With git, the repository and working copy are the same thing, which makes my experience simpler.
We use SourceGear Vault, which has great integration with Visual Studio, and is free for a single user. Depending on what framework and languages you're using, though, Subversion is a great free solution.
First of all, please read these articles by Eric Sink. Eric sink runs a company that creates a Source Control system called Vault. He explains in a newbie friendly manner how to do source control, best practices etc:
Introduction to Source Control
I found it invaluable when I first wanted to understand Source Control.
SourceGear Vault is FREE for a single user. It's interface is intuitive and integrates well with Visual Studio.
If it's just you, you might want to try Bazaar. It's distributed like Git (so it'll be nice for a single person--no server to deal with), but one of their main goals was to make it it much easier to use than Git.
Also, there is a handy gui tool that should make it amazingly easy to use called ToroiseBzr. http://bazaar-vcs.org/TortoiseBzr
There is in fact such a tool. It is called emacs.
Just create yourself a "~/.emacs" file and put the following lines in it:
(setq kept-new-versions 5)
(setq kept-old-versions 5)
And then restart emacs.
This tells emacs to save your 5 oldest and 5 newest versions of that file. They will be kept in files named filename~n~ where "filename" is your file's normal name, and "n" is the backup number.
If you develop your project alone (don't need ane server for collaboration) Mercurial might be you system of choice. I personally value one of its features: it only uses one place to save its information, it is the .hg directory in the root of your project. It doesn't put its data into every directory (like SVN). This way the archive and the project directory is easy to manage.
I've used Visual Source Safe, Perforce, and Subversion. They were all fine, but I would have to say that the support and extensions for Subversion just seemed slightly better. If you're planning on entering/staying in the software industry, you MUST know the fundamentals to source control, and I would highly recommend setting up one of the source control services. Subversion would be my recommendation and is free as well. It will be complicated at first, but you really should use a SVN client to add a GUI to increase utility and cut down on all the complication you're observing.
I quick google of "dreamweaver svn" reveals that many people are working with Subversion in Dreamweaver. I'm a advocate of version control, and SVN in particular, so I would recommend you look into that :)
If you don't want to use a full on version control system (as noted above), you may be able to improve your lot by refining and automating the procedure you described originally. Depending on your comfort with the tools you should be able to put together a script in DreamWeaver itself or in Windows Scripting ( Powershell, VBA, Perl, etc ) that will at least make date-named copies of the folder you are working in every so often. This will keep you from having to do it and make sure there aren't any typo-related problems. Further down that path you can have your script put a copy of your work on a backup drive or remote server, and then you'd have a back up, too.
I'm afraid I don't know much about DreamWeaver, but if it has much scripting support built-in you may even be able to "hook" into the Save/ Auto-Save functions and have them do exactly what you want.
Hope this helps,
adricnet

Resources