I'm actually working on an AngularJS app which consume a lot of independant resources.
The code and the resources are versioned with Git, and the resources (images, html, json...) are organized in module by theme.
Here is my problem : The resources use a lot of disk space in our Git repo.
So do you know any free versioning tool more efficient than git to store this type of file ?
Thanks
Most text based files (HTML, json) should be fine with git. They won't take up too much space. For binary files, such as images, that are changed often you might want to consider using Git Large File Storage. This should limit the space used by old revisions of binary files to the Large File Storage itself, not the development machines.
Related
I've been looking into using Preact to build embeddable/distributable widgets, and one of the benefits that is touted everywhere is that the build size is supposed to be tiny - just 3kb, as it says on their homepage.
But if you look at the NPM packages, Preact says Unpacked Size: 1.26MB while React says Unpacked Size: 291kb. Which to me seems like React is almost 5 times smaller than Preact.
Then, if you look at this data gathered on the size of JS Frameworks, it says that Preact when minified is 16kb. It does say 4kb gzipped, but as far as I know the gzip compression isn't something included as standard in the build process for frontend frameworks, so I don't know where or how Preact got that 3kb figure.
Can anyone tell me how the package sizes listed can differ so much?
You're confusing size on disk of the module off of NPM and bundle size.
The distributed library contains a lot of code that you will not use at one time. It is distributed in a number of module formats for each subpackage (we'll get to that in a second) so you always have a format available to you that works in your environment, be that UMD, ESM, etc. These are essentially duplicates of the code, which increases the size on disk without ever being relevant to the bundle size.
The distributed files also contain a lot of content that is dev-only, like the source code and sourcemaps. Sourcemaps, if you're unfamiliar with them, help you debug built (minified and/or mangled) code. You'll likely disable sourcemaps in your production site (though this is a matter of opinion, there certainly are reasons to keep them enabled) so these files are largely irrelevant.
So Preact comes with a number of submodules, or packages that ship alongside the main library. preact/compat and preact/hooks are examples of this. If you're not using what the submodule offers (full React compatibility in the case of the former, hooks for the latter) then all of that content is irrelevant as well. Each submodule also includes the source code, source maps, and different module formats. These are distributed with the main library for the user's ease of use; no need to install multiple packages.
Lastly, the magic of treeshaking and bundlers. You likely won't use every last piece of code that Preact ships with and bundlers will accordingly remove those unused pieces. This of course does not apply in bundler-less scenarios, but it's pretty common nowadays to use a bundler.
as far as I know the gzip compression isn't something included as standard in the build process for frontend frameworks
Gzip, and more recently Brotli, absolutely is a standard in serving web content. Doesn't matter what the size on the disk is, it matters what gets sent over the internet on request. Preact is roughly ~4kb.
If you'd like a quick peek at what Preact distributes (and what is making up that distribution size), check this out. Lets you inspect the distributed files of the package.
I'm feeling stupid, but I want to know how GitHub and Dropbox store user files, because I have a similar problem and I need to store user's project files .
Is it just like storing project files somewhere in the server and refer to the location as a field in the database, or there are other better methods ?
Thanks.
GitHub uses Git to store repositories, and accesses those repos from their Ruby application. They used to do this with Grit, a Ruby library. Grit was written to implement Git in Ruby but has been replaced with rugged. There are Git reimplementations in other languages like JGit for Java and Dulwich for Python. This presentation gives some details about how GitHub has changed over the years and is worth watching/browsing the slides.
If you wanted to store Git repositories, what you'd want to do is store them on a filesystem (or a cluster thereof) and then have a pointer in your database to point to where the filesystem is located, then use a library like Rugged or JGit or Dulwich to read stuff from the Git repository.
Dropbox stores files on Amazon's S3 service and then implements some wrappers around that for security and so on. This paper describes the protocol that Dropbox uses.
The actual question you've asked is how do you store user files. The simple answer is... on the filesystem. There are plugins for a lot of popular web frameworks for doing user file uploads and file management. Django has Django-Filer for instance. The difficulty you'll encounter in rolling your own file upload management system is building a sensible way to do permissions (so users can only download the files they are entitled to download), so it is worth looking into how the various framework plugins do it.
When working with Google Cloud Endpoints in a appengine project (eclipse based), some files that describe the api are automatically generated every time the endpoint is edited and for every version.
The files are *-v1.api, *-v1-rest.discovery and *-v1-rpc.discovery (the version number may change) and are placed in WEB-INF.
¿Should these files be committed to source control?
My impression is that if the files are automatically generated, they will always be available and there is no need to track them.
Even if I add more versions of the endpoint in the future, I need to keep all those versions for backwards compatibility so all .api and .discovery files will also be generated for all the versions.
Personally, I don't version control (or even worry about backing up) any generated files. I only worry about source and published binaries. And in theory you don't need the binary either because you should be able to recreate the binary from those source files.
I.e., we have a 20MB bzip2 sql file of development data that we'd like to have versioned along with our development code.
However, we don't want this file pulled down from the repo by default with every fresh clone/fetch.
One solution seems to be storing this large file in a separate repo and then link to it with a submodule. Then, a developer would fetch the db file only when they need to retrieve and reset their development database. And then, when there's a schema change, the database file would be updated, committed to the external repo, and the submodule updated.
Is this a good development workflow? Or is there a better way of doing this?
EDIT: The uncompressed SQL dump is 360MB.
EDIT: Github says "no", don't do this:
Database dumps
Large SQL files do not play well with version control systems such as
Git. If you are looking to provide your developers with the most
recent production dataset, we recommend using Dropbox for sharing
files like these among your developers.
I ended up making a simple web server serve the schema dump directory from the repo where dumps are stored. The repo grew really quickly because the dumps are large, and it was slowing people down just to clone it when they had to bring up new nodes.
Here's my requirement:
1. I want my installable to have a custom license agreement
2. run another package as part of the installation
3. let the user have an option of running the app on start-up
What should I use, create a dmg or use PackageMaker available with xcode? Are there any good web pages showing how to use PackageMaker?
Thanks.
They serve different purposes:
DMG (disk images) is just a container file format to solve age-old issues with multi-fork files and transfer protocols and intermediate hosts that can't handle them, by not relying on them in the first place. In addition, the disk images can use internal compression. Your users will thank you for not confusing them with file wrapped in file wrapped in file (although disk images themselves take some explaining initially).
PackageMaker is a full-fledged installer package builder. You can customize the installation process and locations, do sub-installations and pretty much anything else you could possibly need. If your installation entails more than just dragging and dropping an application bundle into place, this is the one to go with.
From your requirements, the choice seems obvious. Since an installer package is itself a bundle, I'd say: create an installer package with PackageMaker and put it in a compressed disk image. Distribute the disk image to your users. It just provides a nicer experience.