I use the free license of Perforce on my home network just to ensure version control and management of home coding projects, university thesis stuff, and general files that I just want to ensure I have previous versions of. It’s a really nice part of the Perforce licensing model that I can get two users and five clients for free. My network has my main PC (with Perforce depot), my personal laptop, my wife’s laptop, and my work laptop all connected. It’ll get worse when my four-year old starts wanting to source control things from his PC!

Having been responsible for some pretty heavyweight Perforce installations in my working life, I did not want to forget the golden rule of checkpointing and backing up. While it would not be a disaster if my Perforce data got lost or messed up, it would be an inconvenience. The scripts published on Perforce user forums, however, tend to be focussed on industrial installations and can be quite complex. For the home it is nice to have something more simple.

So the script here is really aimed at the home user. I don’t use Perforce everyday at home, and so a nightly checkpoint and backup is a waste of time. I tend to run it every two to four weeks. My environment is that I have my main home PC running Perforce, and hanging off this on a USB-2 link is a Western-Digital MyBook external drive that shows itself as D: . Sitting in the Perforce root folder (where the P4 db.* files are, and defined by the P4ROOT environment variable), I have a simple batch file called “backup.bat” (see code below).

The basic backup mechanism is this:

1. Stop the Perforce server
2. Take a checkpoint
3. Copy checkpoint to a location on the MyBook external disk
4. Copy checkpoints to a location on the main PC – just duplication for safety
5. Delete checkpoints from P4ROOT
6. Use Robocopy to incrementally backup all depot files for each depot
7. Restart Perforce server

Here’s the script. Just copy it into a .bat file put it into the P4ROOT folder. To run it, either do as a scheduled task or – as I do – just run it manually at whatever frequency you are happy with.

REM  *** Backs up Perforce
p4 admin stop
p4d -jc -z
copy *.gz d:\Perforce\checkpoints
copy *.gz old_checkpoints
del /Q /F *.gz
rem *** now backup depot files. See following text for explanation
robocopy depot d:\perforce\depotfiles\depot /S /E /MIR /R:1 /W:5 >robolog-depot.txt
robocopy perforce d:\perforce\depotfiles\perforce /S /E /MIR /R:1 /W:5 >>robolog-p4.txt
robocopy work d:\perforce\depotfiles\work /S /E /MIR /R:1 /W:5 >>robolog-work.txt
rem *** Restart server
net start Perforce

The three robocopy lines reflect that I have three depots on my server – ‘depot’, ‘perforce’, and ‘work’. Change these to reflect your own depots.

Robocopy is a free utility that runs under Windows that provides extremely flexible folder copy functionality. See the Wikipedia entry for more details. Just make sure it is either on your search path, or place the EXE in the P4ROOT folder.

The options given to Robocopy just tell it to only copy files that are different to those that already exist. In otherwords, it synchronises one folder structure to another. I pipe the output into a log file so I can see exactly what new files have been copied if I need to.

And that’s it. Dead simple – will take you two minutes to set up, but could save you hours of hassle should something go awry in the future.

One of the words that keeps coming up when I have conversations about new products is that the API must be “powerful”. While intuitively this sounds good – after all, who wants to develop an API that is a bit wimpy – how can you measure and judge the power of one API over another? I’m pretty sure that you can’t assign a wattage to a function call.

Say we have a scene graph that lets a caller insert objects into a hierarchy. An object can reference other objects using a standard instancing metaphor. Lets say this manifests itself like this:

// Add child node to parent
int SceneGraph::AddObject (Node *parent, const Node *child);

Such an API makes it easy to put in a circular dependency, which would cause subsequent scene graph traversals to loop ad-infinitum. So it would be trivial here to make the method check that parent and child are not the same. But what if parent is indirectly reference by child? E.g.

start with A -> B -> C
call SceneGraph::AddObject (C, A)

So, if AddObject were to be “powerful”, would it check for this and return an error? It would certainly make the method robust, as you could not poke invalid data at it. But does robust==powerful?

I would say no. Because even though it stops a user doing something daft that could be hard to diagnose, it does not scale. If the code has to traverse the graph every time a new node is added to the scene graph, then the act of building a model with hundreds or thousands of objects will execute in O(n^2) time complexity. By making this API robust, you actually reduce its power by imposing an overhead upon the user.

Most scene building code will be well behaved – there will be enough knowledge at the application end to “know” that the data is good. And for those times when the application does not know, we can supply a method like:

// Check if one object is a child of another
bool SceneGraph::IsChild (const Node *n0, const Node *n1)

Application code can then query only when needed. The “power” here is in enabling the user to make the decision on balancing cost against convenience.

There’s a good article by Scott Guthrie here that describes the basics. Just scroll down to Tutorial 5 for the INSERT specific bit – it is pretty straightforward and there is no point my repeating it here.

One point to note, however, and why I wrote this particular post. The key step in getting this working is changing the query type from NonQuery to Scalar. But this can get reset by Visual Studio if you choose the Configure option on the ObjectDataSource’s smart tags to modify the query in any way. If you don’t notice this, then suddenly the identity value stops getting returned and your code breaks.

The second tip is that by default the type returned by the insert query is decimal. This can cause some extra casting in your code if you were expecting it to be int. To fix this, just modify the SQL in Scott’s original article to be this:

SELECT CAST (SCOPE_IDENTITY() AS INT)

Here is a usful snippet if you want a way to get the number of rows in each table in your database. I use this in an admin-only page of the web application to provide some at-a-glance statistics. It is also really useful in unit tests for checking the correctness of business logic that may create new entries in several tables in one transaction.

SQL snippet

The bit of SQL is quite straightforward, once you delve down into the depths of SQLServer’s system tables:

SELECT
   so.name AS TableName,
   MAX(si.ROWS) AS [RowCount]
FROM
   sys.sysobjects AS so
INNER JOIN
   sys.sysindexes AS si
   ON OBJECT_ID(so.name) = si.id
WHERE
   (so.xtype = 'U')
GROUP BY so.name
ORDER BY TableName

Presenting in a page

There are a number of ways of getting this into an ASP2 page. I chose to do this via a new table adaptor, and to wrap the query in an Administration class in the business logic layer.

Paste the SQL snippet from above into the query builder to create a new table adapter on the data designer – I have a separate “admin.xsd” to keep these non-user level types away from the core application data types. By default you will get an adapter called “sysobjectsTableAdapter” as it picks its name up from the first table in the query. You can change this if you need to.

You can now go direct to an ASP page by pointing an ObjectDataSource directly at this table adapter, or you can wrap it in an intermediate layer.

Unit test example

My unit test framework has a database reset mechanism that empties all tables and then creates a small set of known test data using the business logic layer (BLL). This empty and reset is done in the test fixture setup, and the subsequent raft of tests use a variety of methods to valid the integrity of the data.

A key test is ensuring that the expected number of rows in various tables have been created. This is particularly important as there are a number of linking and cross reference tables that are not directly seen by a user, but are there for the efficiency of the schema. So checking all these have been set correctly after the BLL has done its work is vital.

In the test class I have something like this :

[TestFixture] public class DBReset
{ ....
   private Dictionary TableRowCounts  = new Dictionary ;
   ...
    [TestFixtureSetup]
    public void FixtureSetup()
    {
        TableRowCounts.Add ("Addresses", 6);
        TableRowCounts.Add ("Agencies", 1);
        ...
    }
}

This sets up a list of all the tables I am interested in and maps the expected number of rows for each table after my fixture setup has initialised the database with known data.

I then have the following unit test as part of this fixture:

 [Test]
 public void CheckRowCountsExpected()
 {
    // Verifies that the correct number of rows in each database
    // has been created as a result of the reset and test
    // data initialisation.
    AdminBll abll = new AdminBll();
 
    Admin.sysobjectsDataTable tabs = abll.GetTableRowCounts();
    foreach (Admin.sysobjectsRow trow in tabs)
    {
       if (TableRowCounts.ContainsKey (trow.TableName))
       {
          //Only check the tables specified in the dictionary
          Assert.AreEqual(TableRowCounts[trow.TableName],
             trow.RowCount,
             "Wrong rowcount for table " + trow.TableName);
       }
    }
}

I also have a second test that verifies the admin query against each table one at a time using direct SELECT COUNT queries in SQL.

To aid my unit testing I needed to populate a database with a load of test data. This data needs to be generated by some of the less technical guys on the project and so exporting CSV (Comma Separated Values) files from Excel seemed a sensible approach. So I then cast around for any existing code that would save me having to write a robust CSV parser. The search turned up this on CodeProject:

LumenWorks.Framework

The neat thing about it is that you can get field values either by index position or column name, assuming you put column headers in your file. The column name feature really helps out with maintainance of the test data as we evolve and vary the schema slightly – we can insert columns with no worries about upsetting existing reader code.
It’s easy to use and install in your own project, and the CodeProject article describes the basic usage pattern. At the moment it is just linked in to the unit test framework, but is so useful I will probably make it a part of the core application libary to facilitate importing data from customers as we anticipate some of our customers could have thousands of records to import when they move to our system.

Just a quick note about a little problem I found using Nunit (2.2.9, but affects earlier revisions too).

After a bunch of edits to my project, I suddenly started getting messages from NUnit GUI “could not load file or assembly nunit.core”. Try as I could, this would not go away. This included everything from reinstalling NUnit to tracking DLL loading via SysInternal’s Process Explorer tool.
My project is non trivial – a top level solution containing an ASP2 web project, a couple of library DLL’s for the business logic, a unit test executable, and a testing helper library to help cross reference test results against direct SQL calls. As I’ll be writing about in another post shortly, incorporating unit testing into an ASP project is a pain in the rear and so I had been restructuring things. My natural assumption based on the error message was I had messed up something in the linking/referencing.

But it turned out to be an error in the application config file – for some reason NUnit reports this as a DLL missing, not that you have mistyped a bit of config text. I simply had a closing tag in the wrong place – still syntactically correct, however.
This link is by someone else who hit the exact same problem, and he has written about in more detail.

So remember folks: never assume a system generates the correct error message for all circumstances.

When setting up a server it is often useful to enable one of the logging levels. The admin guide states that you need to do this:

• p4 set P4DEBUG=server=2

which sets the server logging level to 2. You then need to restart the Perforce server.

However, if Perforce is running as a service (the default under Windows), then this does not work. The reason is that “p4 set” only sets the variable in for HKEY_CURRENT_USER in the Windows registry. However, when running as a service, p4d picks up its environment variables from HKEY_LOCAL_MACHINE. To do this you need to set the ‘-s’ flag. E.g.

• p4 set -s P4DEBUG=server=2

This is still work in progress, but I want to incorporate the user role management tables within my own database, as the concept of users and their distinct roles ties right in with the core business model of the new site.
The following link explains the standalone tool to create the tables:
ASP.NET SQL Server Registration Tool

Invoking the exe runs a wizard requesting the database service – it needs to match the connection string as mentioned in the gotcha above. Since no parameters are specified, it then creates a database “aspnetdb” with all the role tables. This is not quite what I want, as I need these added to my application database.

The answer is to use the command line:
aspnet_regsql -S -E -A all -d

If you run SQL Management Server Studio Express before and after – not forgetting to hit ‘refresh’ – then a bunch of new tables are added: aspnet_Applications, aspnet_Roles, aspnet_SchemaVersions, aspnet_Users, aspnet_UsersInRoles etc. Now it is just a matter of hooking those up with the application tables.

Since I wanted membership and roles as well as basic authentication I used the “-A all” option to add everything in. Of course I get profile and web-part support too with this, but I may choose to exploit such features in the future.

This one gets me every time. When you set up a new data connection to a database running on a local copy of Sql Server Express, in the “Add Connection” dialog you need to prefix your machine name to the defaulted value of “SQLEXPRESS”. E.g. if the network name of your PC is called “devpc”, then the connection string is:

• DEVPC\SQLEXPRESS

While this is the obvious choice and works for many circumstances, a subtle variation is to use the following instead:

• localhost\SQLEXPRESS

This now points the connection to an instance of SQL Server Express running on your local machine. If working in a team where each developer needs their own local copy of a database, then setting this in the web.config file can be a real help.