With the release of StoragePoint 3.1 (...coming in October, public Beta available September 7th) we will deliver our Fileshare Librarian module. It leverages the Shallow Copy Migration capability at the core of StoragePoint to import fileshare content into SharePoint without actually moving it. Aside from being pretty cool stuff, it works incredibly well. Incredibly well = millions of documents and terabytes of content cataloged in a day. This is obviously a vastly superior approach compared with traditional Fileshare Migration solutions that carry the BLOB around as part of the migration workload and ultimately dump them in a content database. And our Librarian (...go ahead, think of the old lady walking around the library cataloging the books and putting data on where to find them in a card catalog...it's a good analogy) doesn't just import the files, it creates a user-definable structure inside of SharePoint to hold it all. You're not pointing a document library at a fileshare and everything lands in one doclib...you can make the destination a Web Application and it will start by creating site collections based on the top level folders in the fileshare, then sites, then libraries, then folders. Start at a site and it will start with lists, then folders. You get the idea. You can also specify site and list templates (where applicable), including custom ones. So if you want the libraries created a certain way just customize a library, save it as a template, and then select it when you setup the Librarian. We also enable both "one and done" and "co-exist" usage scenarios.
"One and done" is pretty straightforward. You have a share, you want to get that content into SharePoint, turn off end user access to the fileshare, and have everyone access the content from SharePoint. While the second scenario is not one we would endorse for use with a traditional "end-user-collaborative" (...dynamic, active, non-static, etc.) fileshare, the product supports it nonetheless. You can run the Librarian on a schedule and it will recognize new files and catalog them accordingly. I think this capability is more useful for a "drop folder" solution pattern. We regularly run into customers that are dropping PDF files from a mainframe print stream (...think monthly statements for a financial services firm) to a fileshare and them importing them into a CM/DM solution. The Librarian would accommodate this usage scenario quite nicely.
Still not convinced? I took a fileshare with 100k items (over 2400 folders), 250GB in size, and cataloged it in 1 hour! The resulting structure in SharePoint was 4 site collections (Finance, HR, IT, and Sales), a whole bunch of sites, an even larger number of libraries, and the rest being folders. Seriously, it took 1 hour on a small farm with commodity hardware and storage! The resulting content database (...it was empty to start) was under 5GB. And this is where the size doesn't matter part comes in. I took the same setup, quadrupled the size of the fileshare to 1TB (...without increasing the number of files) and it still took 1 hour...and the resulting content database was still under 5GB. It could be a share full of GB+ ISOs or multi-100MB videos or spy satellite photos or UFO evidence (...each license of StoragePoint comes with a foil hat) ...it won't matter because the size of the content being shallow copied has no impact on the amount of time it takes to complete that process or the size of the resulting content database. As a note, we're performing some larger scale tests this week on a beefier setup and will publish those results when they are available.
One last feature that I personally like...Simulate! You can simulate the process to weed out any problems before you run it for real. This gives you an opportunity to condense folder and filenames that go beyond SharePoint's URL size limit and/or remove invalid characters and sequences in container and filenames. You can also have the Librarian clean folder and file names automatically...it's an option.
While the Beta we are making available on September 7th is "public", we are limiting the number of folks that get access to it, so get your name on the list by sending an email to info@storagepoint.com. It will be largely 1st come, 1st serve with existing customers and partners getting preference. We'll also be firing up some webinars in the coming weeks to start demonstrating this capability, so stay tuned for that.
As far as licensing goes it will be licensed per WFE and for a very limited time will be included with the core StoragePoint license for no additional charge. So if you're working on a StoragePoint purchase, wrap it up and get this for free, or if you need this capability but haven't engaged us yet an email to info@storagepoint.com can start the process. You can also go to storagepoint.com and try our "sometimes live" chat or request additional information.