Blog - comments

Hi goog article can we install avdf firewall with flat network if it's possible please let me know ?

Thilina
First, thank you for your interrest in this blog.Yes, the byte code will be interpreted each time bu...
BIEHLER Stephane
Pretty sure this is wrong:> already said that JVMs interprets the generated byte code - that's true...
Gs
Michael, great article, however, I would disagree on DRS/Host Affinity. You are legally only requir...
David Bradshaw
Hi lauri, db_file_multiblock_read_count is still used in exadata smartscan because it defines the si...
Blog Stephane Haby SQL Server 2014 - New Features: xVelocity memory optimized columnstore index

dbi services Blog

Welcome to the dbi services Blog! This IT blog focuses on database, middleware, and OS technologies such as Oracle, Microsoft SQL Server & SharePoint, EMC Documentum, MySQL, PostgreSQL, Sybase, Unix/Linux, etc. The dbi services blog represents the view of our consultants, not necessarily that of dbi services. Feel free to comment on our blog postings.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that have been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.

SQL Server 2014 - New Features: xVelocity memory optimized columnstore index

Some customers or colleagues recently asked me about the SQL Server concept xVelocity and in particular the SQL Server 2014 feature new xVelocity memory optimized columnstore index. So I decided to write a few words on the dbi services blog.

 

What is xVelocity?

Introduced in SQL Server 2012, xVelocity is composed of two features:

  •  xVelocity in-memory analytics engine
  •  xVelocity memory-optimized columnstore index

 

xVelocity in-memory analytics engine

The xVelocity in-memory analytics engine is an update of VertiPaq - introduced in SQL Server 2008 R2.

VertiPaq is composed of PowerPivot for Excel 2010 and PowerPivot for SharePoint 2010.

The main goal is to achieve breakthrough performances in analytic queries.

To achieve this aim, the engine uses columnar storage, state-of-the-art compression, in-memory caching, and highly parallel data scanning and aggregation algorithms.

Since SQL server 2012, this feature supports Business Intelligence (BI) - self-service (PowerPivot) and corporate (tabular mode in SSAS).

In the first case with BI self-service, an employee in a company can directly use PowerPivot for Excel to integrate data from different data sources.

In the second case with BI corporate, BI developers or/and IT Pros can use SSDT (SQL Server Data Tools) to create Analysis Services Tabular projects and deploy them on a server so that users can interact with the consolidated Data via Excel and Power View.

 

XVelocity memory-optimized columnstore index

xVelocity memory-optimized columnstore index is a feature which enables high performance query processing especially in Data Warehouses and Data Marts.

Unlike "Standard" indexes, columnstore indexes store data in memory (as the name memory-optimized columnstore index suggests) and the storage organization is not row-oriented.

The characteristic is that data is stored column-wise in compressed form. In SQL Server 2012, the first step was very limited and the main problema is that we cannot UPDATE or INSERT in a table with Columnstore Index.

For example, to update a table with columnstore index, the way is to drop the columnstore index, perform the update operation and then rebuild the index ... which is not fun at all!Cry

Now, in SQL Server 2014, this is possible without dropping the index.Laughing

 

How is it possible?

A new concept / term is introduce in SQL Server 2014 for columnstore: The deltastore.

Deltafile is introduced with the In-Memory OLTP feature and deltastore with the columnstore index.
Yes, SQL Server 2014 is a real delta force!Cool

The best way to define it is to give you the MSDN definition:

"A deltastore is a row-store table that holds rows until the number of rows is large enough to be moved into the columnstore."

They are other similitudes with the in-memory OLTP feature - let's explain this.

 

Insert process

SQL Server inserts rows into the deltastore.

Rows accumulated in the deltastore are stored in rowgroups. Each rowgroup holds from 102,400 to 1,048,576 rows.

When the maximum number of rows in the rowgroup is reached, this rowgroup is marked as "CLOSED".

The "tuple-mover", a background process finds the closed rowgroup and moves it into the columnstore.

b2ap3_thumbnail_InsertProcess_ColumnstoreIndex.jpg

tips.png A SELECT query is an addition between non-closed rowgroups in the deltastore and the columnstore.

 

Delete process

To delete a row, there are 2 cases:

  • In the columnstore, the row is marked logically as deleted, but not physically deleted. A rebuild is necessary to definitively delete the row.
  • In the deltastore, the row is really deleted logically and physically.

 

Update process

To update a row, we have 2 cases - just like for the delete process:

  • In the columnstore, the process is a delete process and an insert process like for the In-Memory OLTP process...
  • In the deltastore, the row is just updated

Here is the MSDN reference for more information.


Type of Columnstore Index

The SQL Server 2014 xVelocity memory optimized columnstore index is composed of two index types:

  • Clustered columnstore index
  • Non-Clustered columnstore index

 

Clustered Columnstore index

Clustered Columnstore index is used for query performance and data compression.

Be careful, a table with a clustered columnstore index cannot have another index. You have two ways to create an index:

  • using a GUI

Select the database, go to the index menu, right-click and select create index and create Clustered Columnstore index:

 

Untitled.png

 

You can see if you can create the index or not:

 

b2ap3_thumbnail_ColumnstoreIndexCreation01.jpg

 

Before the creation of the index, you must delete the others:

 

b2ap3_thumbnail_ColumnstoreIndexCreation02.jpg

 

Now, you can create the index:

  • using script

 

b2ap3_thumbnail_ColumnstoreIndexCreationScript04.jpg

 

Non-Clustered Columnstore index

This is the "old" columnstore index introduced in SQL server 2012.

Like for the clustered columnstore index, you have two ways to create the index:

  • using GUI

Select the database, go to index menu, right-click and select Create index and create Non-Clustered Columnstore index.

You can see if you can create the index or not:

 

b2ap3_thumbnail_ColumnstoreIndexCreationNonClustered01.jpg

 

Prior to the creation of the index, you must select columns.

   

b2ap3_thumbnail_ColumnstoreIndexCreationNonClustered02.jpg

 

And you must delete your previous Clustered Columnstore index:

 

b2ap3_thumbnail_ColumnstoreIndexCreationNonClustered03.jpg

 

Now, you can create the index:

  • using script

 

b2ap3_thumbnail_ColumnstoreIndexCreationScriptNonClustered01.jpg

 

What are benefits of this new xVelocity memory optimized columnstore index?

Making the columnstore index more functional, we can say that the main benefit is this new clustered columnstore index with the ability to continue queries during the updating process without the need to drop and recreate the index.

In my opinion, xVelocity is a very interesting technologies for my customers!

Rate this blog entry:
2

Stéphane Haby is Delivery Manager and Senior Consultant at dbi Services. He has more than ten years of experience in Microsoft solutions. He is specialized in SQL Server technologies such as installation, migration, best practices, and performance analysis etc. He is also an expert in Microsoft Business Intelligence solutions such as SharePoint, SQL Server and Office. Futhermore, he has many years of .NET development experience in the banking sector and other industries. In France, he was one of the first people to have worked with Microsoft Team System. He has written several technical articles on this subject. Stéphane Haby is Microsoft Certified Solutions Associate MCSA) for SQL Server 2012 as well as Microsoft Certified Technology Specialist (MCTS) and Microsoft Certified IT Professional (MCITP) for SQL Server 2008. He is also ITIL Foundation V3 certified. He holds a Engineer diploma in industrial computing and automation from France. His branch-related experience covers Chemicals & Pharmaceuticals, Banking / Financial Services, and many other industries.


MCSA  MCSE  mvp

Comments

  • No comments made yet. Be the first to submit a comment

Leave your comment

Guest Sunday, 26 October 2014
AddThis Social Bookmark Button
Deutsch (DE-CH-AT)   French (Fr)

Contact

Contact us now!

Send us your request!

Our workshops

dbi FlexService SLA - ISO 20000 certified.

dbi FlexService SLA ISO 20000

Expert insight from insiders!

Fixed Price Services

dbi FlexService SLA - ISO 20000 certified.

dbi FlexService SLA ISO 20000

A safe investment: our IT services at fixed prices!

Your flexible SLA

dbi FlexService SLA - ISO 20000 certified.

dbi FlexService SLA ISO 20000

ISO 20000 certified & freely customizable!

dbi services Newsletter