Recent Posts | - May, 2023-5,(1)
- February, 2023-2,(1)
- November, 2022-11,(1)
- July, 2022-7,(2)
- March, 2022-3,(1)
- November, 2021-11,(2)
- August, 2021-8,(2)
- July, 2021-7,(2)
- June, 2021-6,(1)
- May, 2021-5,(1)
- March, 2021-3,(1)
- February, 2021-2,(2)
- January, 2021-1,(7)
- December, 2020-12,(3)
- March, 2020-3,(2)
- February, 2020-2,(1)
- December, 2019-12,(2)
- November, 2019-11,(1)
- October, 2019-10,(1)
- September, 2019-9,(1)
- August, 2019-8,(1)
- May, 2019-5,(1)
- April, 2019-4,(2)
- March, 2019-3,(2)
- December, 2018-12,(1)
- November, 2018-11,(4)
- July, 2018-7,(1)
- May, 2018-5,(3)
- April, 2018-4,(2)
- February, 2018-2,(3)
- January, 2018-1,(3)
- November, 2017-11,(2)
- August, 2017-8,(1)
- June, 2017-6,(3)
- May, 2017-5,(5)
- February, 2017-2,(1)
- December, 2016-12,(1)
- October, 2016-10,(2)
- September, 2016-9,(1)
- August, 2016-8,(1)
- July, 2016-7,(1)
- March, 2016-3,(2)
- February, 2016-2,(3)
- December, 2015-12,(5)
- November, 2015-11,(5)
- September, 2015-9,(1)
- August, 2015-8,(2)
- July, 2015-7,(1)
- March, 2015-3,(2)
- February, 2015-2,(1)
- December, 2014-12,(4)
- July, 2014-7,(2)
- June, 2014-6,(2)
- May, 2014-5,(3)
- April, 2014-4,(3)
- March, 2014-3,(1)
- December, 2013-12,(2)
- November, 2013-11,(1)
- July, 2013-7,(1)
- June, 2013-6,(2)
- May, 2013-5,(1)
- March, 2013-3,(3)
- February, 2013-2,(3)
- January, 2013-1,(1)
- December, 2012-12,(3)
- November, 2012-11,(1)
- October, 2012-10,(1)
- September, 2012-9,(1)
- August, 2012-8,(1)
- July, 2012-7,(6)
- June, 2012-6,(1)
- April, 2012-4,(1)
- March, 2012-3,(3)
- February, 2012-2,(3)
- January, 2012-1,(4)
- December, 2011-12,(3)
- October, 2011-10,(3)
- September, 2011-9,(1)
- August, 2011-8,(10)
- July, 2011-7,(2)
- June, 2011-6,(7)
- March, 2011-3,(2)
- February, 2011-2,(3)
- January, 2011-1,(1)
- September, 2010-9,(1)
- August, 2010-8,(2)
- June, 2010-6,(1)
- May, 2010-5,(1)
- April, 2010-4,(3)
- March, 2010-3,(2)
- February, 2010-2,(3)
- January, 2010-1,(1)
- December, 2009-12,(3)
- November, 2009-11,(3)
- October, 2009-10,(2)
- September, 2009-9,(5)
- August, 2009-8,(3)
- July, 2009-7,(9)
- June, 2009-6,(2)
- May, 2009-5,(2)
- April, 2009-4,(9)
- March, 2009-3,(6)
- February, 2009-2,(4)
- January, 2009-1,(10)
- December, 2008-12,(5)
- November, 2008-11,(5)
- October, 2008-10,(13)
- September, 2008-9,(10)
- August, 2008-8,(7)
- July, 2008-7,(8)
- June, 2008-6,(12)
- May, 2008-5,(14)
- April, 2008-4,(12)
- March, 2008-3,(17)
- February, 2008-2,(10)
- January, 2008-1,(16)
- December, 2007-12,(6)
- November, 2007-11,(4)
|
|
|
|
Moving Blobs
|
9/12/2008 3:29:35 PM
|
|
Moving blobs to a dedicated file group, maybe with several data files, is a very good idea ! Of course, if you wait to do this until your database grows quite large, you've got a problem. I define large as 50+ gb.
Currently I have an issue with a 120gb database. 100gb of the data is in one table, with an image data type. I'd like to isolate this tables blob data to a dedicated file group, made up of 4 data files. This is not difficult, what is difficult is making it happen.
I've experimented with doing this 3 ways:
1. SQL Management Studio - 28 hours.
2. SSIS - 18 hours.
3. Custom TSQL Scripts, 4 spids, dividng the table up, 12 hours.
I have found all of them to be very painfully slow. So slow in fact that I'm not able to do it, as I currently can't take a 12+ hour outage.
Knowing your volumetrics and design up front is very important for this exact reason.
SQL Server 2005 can do this quicker with parrallel loads, if the table is partitioned, but this is a 3rd party database from a vendor and can't support partioned tables.
SQL Server 2008 provides some new options for storing the blobs outside the database, which is always my preference if possible.
Below is the script for option 3, handy sometimes:
use emailxaminer_run
go
/*
select min(fileid),max(fileid) from rt_filecache
-- calculations to figure out each spid to run.
1. 980,066 to 7,553,806 start = min(fileid)
2. 7,553,807 to 14,127,547
3. 14,127,548 to 20,701,288
4. 20,701,289 to 27,275,029 end = max(fileid)
select 27275029 - 980066
select 26294963 / 4 --6573740
-- thread 1
select 980066 + 6573740 -- 7553806
select 14127548 + 6573740
*/
declare @min int
declare @max int
Declare @startPosition int
declare @endPosition int
declare @currentPosition int
set @startPosition = 980066
set @endposition = 30000000
set @currentPosition = @startPosition
declare @increment int
set @increment = 1000
While @currentPosition < @EndPosition
Begin
begin tran
INSERT INTO dbo.Tmp_RT_FileCache (FileID, MessageId, FileType, FileSize, datelastmod, FileName, FileBLOB)
SELECT FileID, MessageId, FileType, FileSize, datelastmod,
FileName, FileBLOB
FROM dbo.RT_FileCache with (nolock)
Where FileId Between @currentPosition AND
(Case WHEN @currentPosition + @increment > @endPosition then @endPosition
Else @currentPosition + @increment END)
commit tran
Print 'Where Clause = ' + convert(varchar(10),@currentPosition) + ' AND ' +
Convert(varchar(10), (Case WHEN @currentPosition + @increment > @endPosition then @endPosition
Else @currentPosition + @increment END))
Set @currentPosition =
Case WHEN @currentPosition + @increment > @endPosition then @endPosition
Else @currentPosition + @increment + 1 END
--Print @currentPosition
End
|
Blog Home
|
|