Tag: version control

Git Bundles – A Beginner’s Guide

On February 28, 2019

I recently ran into a situation at work where I needed to share a Git repo “across an air gap.” In other words, I needed to be able to read a Git repo without actually having read access to the Git repo. I did some research and found out that this can be done with Git Bundles, but when I tried to look for info on how to actually USE Git Bundles, I mostly found a lot of Stack Overflow and Reddit threads with 23 year old know-it-alls providing helpful tips like “That’s stupid. Why would you ever need to do this?” The answer is that you work for a financial services company and they are paranoid to give developers access to anything that exists on the internet. Anyway, enough ranting. I pieced together a method for using Git Bundles that worked well for me and I thought I’d share it.

So here’s the setup:
A company we’ll call “OutsourcePro” is hired to build an application for your employer, who we will call “FullTimeJob.” OutSourcePro uses private Bitbucket repos for all of their projects and they offer to provide the devs at FullTimeJob access to the repos for purposes of doing regular code reviews. Unfortunately, FullTimeJob is a financial services company and finds the internet to be too scary to allow anything like Git repo access. So, FullTimeJob’s developers are told that they can’t have the access.

OutSourcePro offers to send a giant zip file with the entire project at intervals of their choosing (rarely or never). But, FullTimeJob’s developers find out about Git Bundles. FullTimeJob devs tell OutSourcePro to provide a Git Bundle every Tuesday and Friday. This will allow FullTimeJob devs to review the changes without needing to read every single line of code to figure out what’s different. FullTimeJob’s security experts agree that receiving a giant zip file via email twice a week is more secure than accessing a private Git repo directly (um… okay). Great! But, now what? The interwebs have very little useful info on how Git Bundles actually work. So, a developer at FullTimeJob does some experimenting and comes up with a plan:

Method 1: One-way Traffic
This assumes that all development work is being done by OutSourcePro and that FullTimeJob’s developers only need read-access for code reviews and whatnot. Developers at FullTimeJob will not be committing or pushing any code.

OutSourcePro creates a private Git repo (on Github, Bitbucket, or somewhere else).
FullTimeJob creates a corresponding Gitlab repo on their proprietary, enterprise Git server.
OutSourcePro does daily work and pushes ALL changes to to their private Git repo (including work in progress)
On Tuesdays and Fridays, an OutSourcePro developer pulls ALL commits from ALL branches to their local repository.

The OutSourcePro developer creates a git bundle:

git bundle create outsourcepro_DDMMYY.bundle –all

The OutSourcePro emails the bundle to FullTimeJob
A FullTimeJob developer copies the bundle to their Desktop and clones it to a new folder:
```
git clone /Users/FullTimeJobDev/Desktop/outsourcepro_MMDDYYYY.bundle
```
This will clone it into a folder named for the bundle: outsourcepro_DDMMYYYY
The FullTimeJob Developer should then point the origin of the new local clone to their private enterprise Git server:
```
git remote set-url origin https://ftjgitlab.com/project/repo-name.git
```
Once the origin has been set, the FullTimeJob developer can push all changes from the local clone to the enterprise remote.
Once all the changes have been pushed, the local clone should be deleted from the FullTimeJob developer’s Desktop.
Now, all FullTimeJob developers can fetch/pull the changes from the enterprise remote to their local repos.

Basically, this process allows you to make a duplicate of the Git repository on your own server and keep up with the changes being made by the OutSourcePro team.

Method 2: Two-way Traffic
What if developers on BOTH teams needed to make changes to the code? This use case is actually very similar, with both teams sending Git bundles to each other and following the process above. But there are a few tips I will suggest to make it easier:

Don’t cross the streams! Before starting the project, decide what each team will name the branches on their Git repos, and make sure that the two teams are ALWAYS USING SEPARATE BRANCHES. Same-named branches will cause chaos. In our example:
- FullTimeJob will eventually own all of this code, so they name their main branches “dev” and “master.” All other branches on their end will be named “ftj-something.”
- OutSourcePro will eventually hand over all of the code, so they name their main branches “osp-dev” and “osp-master.” All other branches will also be named “osp-something.”
Decide who will do the merging into the final repo branches. Again, in our case:
- FullTimeJob will own the code and the final project will reside in the “dev” and “master” branches create by FullTimeJob. So, only FullTimeJob developers are allowed to merge code into these branches.
- Similarly, OutSourcePro owns all of the “osp-” branches and only OutSourcePro devs are allowed to merge anything into those branches.
Both teams should be merging the other team’s work into their branches regularly and resolving the conflicts.
Nail down specific times when new bundles will be delivered to each team so that the syncing and merging can be done at scheduled times.

That’s it! I suggest testing this out internally before you try to do it in a real-life scenario. Have two developers act as members of the two teams. One developer can create a git bundle and send it to the second dev, who will follow the process of trying to duplicate it to a new repository. Then, have the first dev make some commits and send another Git bundle to the second dev who tries to add the changes to their duplicate repo. Good Luck and let me know if I missed any important details!

Converting SVN repos to Git (for dummies)

By Andy

On August 29, 2014

In Uncategorized

We used to use Subversion (SVN) for all of our source control at work a few years ago. Then, we switched over to Git a while back. I had several old projects still stored in our SVN account at codebasehq.com and I needed to migrate them to Git repos. A few of them were work projects that needed to be moved to Github. Others were personal projects that I wanted to store on BitBucket. I spent several hours combing through blog posts figuring this out, so I thought I’d make a “dummies” guide on how to do this for people like me who are not Git experts. For this conversion, you will need:

Sourcetree – a great, free Git GUI. Get it here
Git installed on the command line. Sourcetree has it’s own Git client, but we will need to do some command line stuff. Unfortunately, I can’t offer any tips on setting up Git. I did it several years ago and didn’t make any notes on it.
A mac. Sorry, but these instructions are for mac users. They will be very similar for PC users, but may need slight modification.

Once you have those things, here’s what you do:

Create a folder on your machine where you will store your files during this process. I created one called “SVN_TO_GIT” on my desktop.
Log in to your SVN account and find the SVN repo you want to convert. browse through the files and try to get a list of the user ids that have made commits. For example, in one of my work projects, I found the following user IDs:
andytb9
johnboy
andy
john
toolbox
Create an authors.txt file to convert the old svn IDs to git emails. This is really simple. Just save a blank text file in the folder you created above as “authors.txt”. In this file, add a line for each of the user IDs you found in Step 2. These lines specify how to convert the SVN user IDs:
```
andy = Andy Watt <andy@gmail.com>
andytb9 = Andy Watt <andy@gmail.com>
johnboy = John Doe <john@gmail.com>
john = John Doe <john@gmail.com>
toolbox = Andy Watt <andy@gmail.com>
```
**Note how you can actually re-map the project contributors. I could simply assign all SVN IDs to my email if I want.
Create a subfolder in the SVN_TO_GIT folder you already created. This folder will hold the repo that you will convert, so name it something that makes sense. For this example, I will name mine “REPO_1”.
Open Sourcetree and clone the SVN repo into the REPO_1 folder you created above. Clone it as if it was a Git repo, but enter the URL of the SVN repo. You will probably need to enter your user name and password for the SVN account here.
Sourcetree will notice that it’s an SVN repo and ask you for some details on how to convert it:
- Create local repository of type: Git
- Convert from SVN revision: 1
- Author map file: (browse to your authors.txt file)
BTW, after you enter your SVN credentials, make sure that Sourcetree is still planning to save your cloned repo into the REPO_1 folder. Sourcetree has a quirk that sometimes changes this destination folder unexpectedly.
Click “clone”
If you get an error about “Can’t locate SVN/Core.pm”, go here and follow the instructions to get everything hooked up correctly.
Or… if you get a crazy error like this:
```
Can't locate object method path via package Git::SVN at /usr/local/git/lib/perl5/site_perl/Git/SVN/Ra.pm line 338.
```
- go to Sourcetree > Preferences.
- click on the Git tab at the top.
- near the bottom, change the Git version to “Use system Git”
- locate the system Git executable file on your machine. You can do a search in finder for “git” and look for a file with that name – the icon will look like a terminal window.
- that should fix the error
Show the output and check for errors. If it clones successfully, you’re almost done! If not, you probably forgot a user in the authors.txt file. Scan the errors to see if it is bonking on a user id. Add the conversion to your authors.txt file and try again.
Log in to Github, Bitbucket, or the Git account of your choice. Create a new repo. Be sure to specify whether this is a public or private repo. I did not add a readme, but it’s ok if you do.
Copy the url for the new Github repo that you created (in my example: https://github.com/andy/dummy-project.git)
On your mac, open a terminal window
cd (change directory) to the SVN_TO_GIT/REPO_1 folder that now contains your cloned repo.

run the following commands:

git remote set-url origin
git remote set-url origin https://github.com/andy/dummy-project.git
git push origin master

Alternative method if the previous steps didn’t work for you:

cd /path/to/my/repogit 
remote add origin https://tb9andy@bitbucket.org/andy/dummy-project.git
git push -u origin --all 
git push -u origin --tags

After it finishes, log in to github and take a look at your awesome new git repo with all of its commit history. Pat yourself on the back for your awesomeness.
Create a new folder on your local machine and clone the new github repo into it. This is the local folder you should work from. Don’t work from the SVN_TO_GIT/REPO_1 folder you created. You should be able to delete that now.
You should remove the old SVN remote and any local references to it on your machine once you’re sure that everything has converted correctly.

If you need to convert an old SVN repository to Git, I hope this will save you some time. If I missed anything, leave a comment and let me know.

Creating a simple iOS game – Bouncing Babies

By Andy

On March 14, 2013

In Game Development, iOS Game Dev, Uncategorized

The first post in my iOS game development series…

I like to create games that use interesting gameplay mechanics – things I’ve never seen before, or an interesting twist on a classic game. The problem is that new mechanics are difficult to code. If you want to make something standard, like a Super Mario clone, it’s easy to find a game engine that will do most of the work for you. But new mechanics often require modifying a game engine, or even creating one from scratch. This can be time consuming and frustrating.

I’ve been working on a conceptual game prototype for a while now and it has me pulling my hair out at the moment. To make matters worse, this is a “stripped down” version of a larger game that I want to create. I thought this light version of my concept would be easier to code, but it has turned out to be pretty complex too.

In the meantime, I’m interested in creating a game for iOS. So, I’ve decided to create a very simple game from scratch and blog about the entire process. I’ve never created anything for iOS, so I will have a lot to learn. I also plan to post all of my progress publicly on github, so anyone who wants to follow along can pull my code and see what I’m doing. Since this is a learning process, there will likely be a lot of bad code and blind alleys. And since I have a day job and a family, progress is likely to be slow.

With these constraints in mind, I’ve decided not to design my own game entirely from scratch because I will inevitably add some seemingly simple gameplay element that will derail the entire process. Instead, I’ve decided to remake a classic DOS game from back in the day…

Bouncing Babies was a deceptively simple arcade game that I used to play on my first PC – a Tandy 1000 (yes, I’m old). Here’s a video of the game:

Like all classic action arcade games, the gameplay is simple, but it’s surprisingly addictive. It was one of my favorite games when I was a kid and there are few good remakes of it. Considering my time constraints, this is a project that I might actually be able to finish in a reasonable time period. Additionally, the game code shouldn’t be too complex, so development shouldn’t require any weird hackery (famous last words). I can still do a full redesign of the graphics and animation, but the gameplay will be the same.

So, stay tuned for more updates as I dive into this. This whole project may go down in flames, but I promise it will be an interesting ride.

Here’s a link to the github repo I will be using:
https://github.com/wastedpotential/bouncing-babies

Version Control Throwdown – Git vs. SVN

By Andy

On February 7, 2013

In Game Development

Version control throwdown - GIT vs. SVN I see a lot of forum threads and blog posts on “which is better – Subversion or Git? The real question should be “which is better for my workflow – SVN or Git?” I’ll attempt to help you sort it all out.

I’ve been handling version control on my projects with Subversion (or SVN, for short) for the past couple of years. I switched to using Git about 6 months ago and have been using it for version control in most new projects. Git and SVN both have their strengths and weaknesses. Which one you use should be based on the type of work you’re doing.

The way I used to handle version control was simple. Any time I made any serious updates to a project, I duplicated all of the files into a new folder and started working. That way, I still had a copy of the previous, stable version if I needed it. This is a ridiculous way to handle it, but it’s simple and it works, as long as you remember to duplicate the files before you begin working.

SVN is an older model of version control that handles this for you. The files live in a repository on a remote server and you create a local copy on your computer (or even multiple computers). “Updating” pulls the latest version down from the remote server and updates the files on your local version that are out of date. “Committing” pushes any local changes back to the remote server when you are done working. You can revert back to an older version of a project (or even a single file) if you need to. It’s a great way of tracking a project without needing to retain duplicates of every file. It is designed purely for keeping track of project versions – If multiple people try to commit changes to the same files, a conflict occurs, which you must resolve, usually by comparing the 2 files and manually merging them. The goal is to keep your local and remote repos in sync at all times. I used Tortoise as my SVN interface and I’m fairly happy with it.

Git differs from SVN in a fundamental way. It’s a distributed version control system, meaning that it’s designed for multiple developers to work simultaneously on the same project. You again have a remote master repository and a local copy on your machine. The big difference is that when you “commit” a change, the changes are only made to your local repository. Then, when you have finished your work and the local version is stable, you “push” all of your local commits to the remote repository. The idea here is that you can have version control on your local machine and only merge your changes into the master when they are finished and stable.

Git also allows for “branching,” which means that there are different versions of the project stored in different branches. Master is the stable, production branch. Then you could create a branch for a new set of features that you are developing. This way, another developer can make bug fixes to the Master branch while you develop new features on the other one. Then you merge them together later. Git allows for many branches to be created, so different developers can work simultaneously without jeopardizing the stability of the Master repo. I’m not sure why, but Git is also generally faster than SVN for updates and commits. A lot of Git users do everything on the command line – I prefer a decent GUI, so I use SmartGit. Mac users tend to like SourceTree.

So, from my description, it would sound like Git is superior, but that isn’t completely true. It really depends on your project.

Git is designed for large projects with multiple developers working simultaneously. It handles this use case very well. The branches help prevent developers accidentally overwriting each other’s work, or creating instability in the master repository. The downside is that Git has a much steeper learning curve than SVN. I’ve been using it for months and I’m still getting the hang of it.

SVN, on the other hand, is really simple to work with, if you only have one or two people working on a project and the chances of creating repository conflicts are small. So, if you don’t need the “distributed” part of version control, SVN is a great choice.

Another thing to consider is the number of large binary files you will want to have version controlled. Binary files are non-text files, like videos and Flash source files (FLAs). Neither of these version control systems can keep track of changes in binary files, but SVN plays nice with large binary files – Git does not. Git purists argue that large binary files don’t belong in the repository, but I don’t necessarily agree. So, if you plan to have a lot of videos or Flash files in your repository, Git may not work for you. You can tell Git to ignore the large files, but that seems to defeat the purpose of version control.

remote svn repo on a usb flash drive Which do I prefer? Again, it depends. For my projects at work, Git has been great (unless I need to version a bunch of videos, Then, it stinks). I have a handful of personal projects that I prefer to handle with SVN. I have a USB flash drive on my keychain that contains the remote SVN repositories. So, no matter where I am or which computer I’m using, I can update from the USB drive and work, even if I have no internet access. You can do this with Git too, but since I’m the only developer on these projects, I don’t need the fancy-schmanciness of Git and SVN is less hassle.

Hopefully, this will help you decide which version control system will suit your workflow. If you want to set up a USB thumb drive repository, here are simple instructions for setting up a local SVN repository. Here are instructions for doing it with Git.