Qing Zhang's technical cube

Stevey's Google Platforms Rant

2011-11-20T01:55:00.001-05:00

You cannot miss this great and interesting artical about the comparison about platform architecture and IT management! http://steverant.pen.io/

some notes about In Memory Database

2011-07-26T02:56:00.000-04:00

Accessing data in memory reduces the I/O reading activity when querying the data which provides faster and more predictable performance than disk. In applications where response time is critical, such as telecommunications network equipment, main memory databases are often used.

ACID support
Volatile memory-based MMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation. Many MMDBs add durability via the following mechanisms:
- Snapshot files, or, checkpoint images, which record the state of the database at a given moment in time.
typically generated periodically, or, at least when the MMDB does a controlled shut-down. Only offer partial durability.
Full durability need to be supplemented by one of the following:

- Transaction logging,
records changes to the database in a journal file, facilitates automatic recovery
- Non-volatile random access memory (NVRAM),
static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM (EEPROM). Recover the data store from its last consistent state upon reboot.
- High availability (reduncy/replication)
rely on database replication, with automatic failover to an identical standby database in the event of primary database failure.

To protect against loss of data in the case of a complete system crash, replication of a MMDB is normally used in conjunction with one or more of the mechanisms listed above.

--------------

Oracle In-Memory Database Cache Architecture
- shared libraries
It's in contrast to a more conventional RDBMS, which is implemented as a collection of executable programs to which applications connect, typically over a client/server network.

- memory-resident data structures
It is maintained in shared memory segments in the operating system and contains all user data, indexes, system catalogs, log buffers, lock tables and temp space
.
- database processes
a separate process to each database to perform operations including the following tasks:
> Loading the database into memory from a checkpoint file on disk
> Recovering the database if it needs to be recovered after loading it into memory
> Performing periodic checkpoints in the background against the active database
> Detecting and handling deadlocks
> Performing data aging
> Writing log records to files

- administrative programs
Utility programs are explicitly invoked by users, scripts, or applications to perform services such as interactive SQL, bulk copy, backup and restore, database migration and system monitoring.

- IMDB Cache
A cache group is created to hold the cached data. It is a collection of one or more tables arranged in a logical hierarchy by using primary key and foreign key relationships. Each table in a cache group is related to a database table. A cache table can contain all rows and columns or a subset in the related table. Cache groups support these features:
> Applications can read from and write to cache groups.
> Cache groups can be refreshed from Oracle data automatically or manually.
> Updates to cache groups can be propagated to Oracle tables automatically or manually.
> Changes to either Oracle tables or the cache group can be tracked automatically.

When rows in a cache group are updated by applications, the corresponding rows in tables can be updated synchronously as part of the same transaction, or asynchronously immediately afterward. The asynchronous configuration produces significantly higher throughput and much faster application response times.

Changes that originate in the tables are refreshed into the cache by the cache agent.

Each cache group has a root table that contains the primary key for the cache group.
Rows in the root table may have one-to-many relationships with rows in child tables,
each of which may have one-to-many relationships with rows in other child tables.

Each primary key value in the root table specifies a cache instance. Cache instances form the unit of cache loading and cache aging.

The most commonly used cache group types are:
> Read-only cache group: committed updates to tables are automatically refreshed to the corresponding cache tables in the IMDB Cache database.
> Asynchronous writethrough (AWT) cache group: committed updates to cache tables in the IMDB Cache database are automatically propagated to the corresponding tables asynchronously.
> Synchronous writethrough (SWT) cache group: committed updates to cache tables in the IMDB Cache database are automatically propagated to the corresponding tables synchronously.
> User managed cache group

Cache groups can be either dynamically loaded or explicitly loaded.
> explicitly loaded cache groups: the application preloads data into the cache tables from the database using a load cache group operation. .
> dynamic cache groups: cache instances are automatically loaded into the IMDB Cache from the database when the application references cache instances that are not already in the IMDB Cache. The use of dynamic cache groups is typically coupled with least recently used (LRU) aging.

Keep a cache group synchronized with the corresponding data in the Oracle tables:
> Autorefresh
* incremental autorefresh: updates only records that have been modified since the last refresh.
best when updated often, but only a few rows are changed with each update.
* full autorefresh operation, refreshes the entire cache group at specified time intervals.
best if table is updated only once a day and many rows are changed.
> Manual refresh
best if the application logic knows when the refresh should happen.

Aging:
Records can be automatically aged out, usage-based or time-based.

Passthrough feature
checks whether the SQL statement can be handled locally by the cached tables in the IMDB Cache or if it must be redirected to the database.

How to make contributions to Hadoop Hackathon

2011-06-26T10:43:00.004-04:00

Joined the Apache Hadoop Hackathon meeting hosted by the LA Hadoop User Group in Shopzilla last week. Here's a detailed explanation how you can make contributions:
http://bit.ly/ifGaMc
submitting patches, or simply running tests and identifying bugs would be great.

The Amazon's lesson on Cloud Computing

2011-04-22T22:40:00.004-04:00

Read about this article in today's New York Times: Amazon’s Trouble Raises Cloud Computing Doubts. The difficulties EC2 has come across impacted many business on their platform.

In my view, fault tolerance is still the key concern for big companies with critical data. That's the reason that despite the overwhelming platforms and applications based on MapReduce day by day, big banks or retailers still prefers traditional data warehouses such as Teradata, as it provides more safety for their data storage (another important reason is the strong query optimization and efficient data analytics).

In the long run, I believe failures will be handled better and better for the Cloud Computing and MapReduce technologies. We have seen the same path for online trading platforms: when they're coming out initially, system breaks here and there. But after several years, almost all people are doing their trades online. The current difficulities with Cloud and MapReduce will also go away as time goes on. Traditional data warehouse companies also believe in this. Teradata, DB2, Exadaa, GreenPlum,... all of them are investigating, or already proposed some hybrid models combining cloud and MapReduce into their product.

For now, it's still strongly recommended that business should maintain some backup system/storage on their own, before the cloud is resilient enough. And for the cloud service providers such as Amazon, they should give higher priority to fault tolerance system design. If they cannot provide 24/7 at this time, at least they should make the recovery more quickly and transparent to customers. In the current event, the impact has lasted two days. This is just way toooo bad.

Facebook finally launches social deal

2011-04-17T15:22:00.004-04:00

Refer to this link: http://gigaom.com/2011/04/15/how-facebook-can-beat-groupon-by-making-deals-social/.
Just as I expected in my last Dec.'s post, I thought Groupon is an easy to duplicate model, and should join Fackbook instead of being bought by google or going independent IPO. Now it seems it's missing this opportunity as Facebook already started its own social deal. The deal sharing within a social context would be more effective: anyway, friends often have similar interest and taste, and recommendations from friends are automatically including trust and reputation mechanisms. As long as Facebook doesn't make stupid mistakes in the future, Groupon will be in big trouble.

Amazing work of object tracking!

2011-04-15T10:14:00.003-04:00

Can't wait for new products adopting this technique!
http://www.youtube.com/watch?v=1GhNXHCQGsM

Opinions about ActiveBase webcast

2011-03-31T14:36:00.003-04:00

I attended the ActiveBase webcast just now. The brief introduction of their product can be found here: link. Here's a quick summary of their product,

ActiveBase, currently support oracle and sql server, provides sql proxies based on query rewriting to dynamiclly anonymize data. There's no need of change to application and db. The product can be installed 1. on each database server. 2. dedicated app servers as proxies/firewall.

The protection solutions they provide: masks, scrambles, hides, audits, blocks fields. They adopt Role Based Access Aontrol to determine the anonymization methods and results returned to user.

Major concerns I have about their product:

- It seems the query rewritting technique needs a lot of manual effort, for each type of query. It is hard to automate for ad hoc queries. So there's big maintaince cost.

- They don't have any privacy guarantee on their solution. Specificly, they use the query proxy technique and dynamiclly rewrite each query and return answers. Will the attacker be able to design queries carefully, and get multi versions of the anonymized data, make comparisons and thus lead to information leakage?

- What's the usability of the anonymized dataset? For simple masking, the data doesn't have much usability after anonymization.

Anyway, I'm still glad to see privacy and anonymization are getting more and more attention, and new products being developed. An interesting future work would be: what are the special techniques needed to provide anonymity to no-sql databases and cloud db?

Data Protection general practices

2011-03-14T22:18:00.002-04:00

Data Protection Layers:
- Application
- Database
- File System

Operation Cost Factors:
- Performance
- Storage: data storage requirements
- Security
- Transparency: change to applications, and supports to utilities.

Data Protection options:
- Clear: actual value

- Hash: unreadable, not reversible
keyed hash(HMAC) provides strong protection
Considerations: key rotation

- Encryption: unreadable, reversible
Considerations: storage type, transparency to applications, key rotation

- Format controlling encryption: unreadable, reversible
Considerations: key rotation.

- Replacement(tokens): unreadable, reversible
Proxy value created to replace original data.
Considertations: transparency for applications needing original data.

Continuous data protection:
- Automatically save a copy of every change made to the data. It allows the user or administrator to restore data to any point in time.

- Advantage: Most continuous data protection solutions save byte or block-level differences rather than file-level differences. So if the portion of write data is small, save only the write changes will require less space on backup media.

- Cost: introduce extra disk write operations and continuous network usage.

Failure Protection in Teradata

2011-03-14T01:05:00.004-04:00

let's first look at Data Allocation in Teradata: Bacisally, OS recognize logical units(LUN), which is composed of slices(UNIX) or partitions(Windows/Linux) from each of the disk drives of a disk rank. Then the PDE translates the LUN into one or more pdisks. psdisks are then assigned to AMPs. All the logical disk spaces an AMP manages is called a vdisk. In general all pdisks from a rank will be assigned to the same AMP.

Failure protection in Teradata falls in the several different levels:

Disk drive level: RAID
RAID: Redundant Array of Independent(or Inexpensive) Disks.
The various designs of RAID systems involve two key goals: increase data reliability and increase input/output performance. There are six different designs RAID 1 to RAID 6 that provides fault tolerance (There's also so called RAID 0 which has no fault tolerance, and RAID 10 TBD.)

Teradata supports RAID 1 and RAID 5.
- RAID 1(mirroring without parity): Data is fully replicated in mirror disk(s). Read blocks from the 1st available disk. Besides failure protection it also provides great performance benefit.

- RAID 5(block-level striping with distributed parity): Data is striped across a rank of disks one segment at a time. Parity is also striped all disk drives, interleaved with data. When a disk fails, data is reconstructed on the fly using existing data and parity.

RAID 1 is faster than RAID 5, as the two(or more) disks are read parallelly, and no parity computation.

AMP level: Fallback tables
Storing a 2nd copy of each row of a table on a different AMP in the same cluster. Specified during table creation. Fallback will cause twict I/O on data modifications.

Obviously the highest level of protection is RAID 1 with Fallback protection.

Componenet/Process Level: Journal
Journals are used for specific types of data or process recovery.
Recovery Journals: maintained by system automatically. Two different types:
- transient journal: keeps "before image" of changed rows so data can be restored to previous state in case of an interrupted transaction. Happens in each AMP.
- down AMP recovery journal: log write changes to data on the failed AMP by other AMPs in the cluster. Then applying changes to the recovered AMP.

Permanet Journals: optional, user specifies at table level, and can store before images or after images to provide full-table recovery to a specific point in time.

Database Object Level: Locks
Applied at 3 different levels: Database/Table/Row Hash
4 types:
- Exclusive: at db/table level, used for DDL, blocks all other locks
- Write: ensures data consistency while writing, only allow access locks
- Read: ensures data consistency while reading, allows read/access locks
- Access: allows table update only for small single-row changes, blocks exclusive locks.
Local deadlocks are checked at AMP level, and global deadlocks are coordinated by PE on a timed basis.

Top 10 Database Security Threats

2011-03-10T01:12:00.004-05:00

Here's a brief digest of the docs from Imperva Inc. Due to copy right I won't post the original pdf here, but can easily search for it online.

- Threat 1 - Excessive Privilege Abuse:
When users (or applications) are granted database access privileges that exceed the requirements of their job function, these privileges may be abused for malicious purpose.
> Prevension - Query-Level Access Control
* restricts database privileges to minimum-required SQL operations (SELECT, UPDATE, etc.) and data.
* most database has some level of query-level access control (triggers, row-level security, etc), but too time consuming to do manually.

- Threat 2 - Legitimate Privilege Abuse
Users may also abuse legitimate database privileges for unauthorized purposes.
> Prevension: Understanding the Context of Database Access
Enforcing policy for client applications, time of day, location, etc., identify users access in a suspicious manner.

- Threat 3 - Privilege Elevation
Attackers may take advantage of database platform software vulnerabilities to convert access privileges from those of an ordinary user to those of an administrator. Vulnerabilities may be found in stored procedures, built-in functions, protocol implementations, and even SQL statements.
> Prevension: Intrusion prevention systems(IPS) and Query Level Access Control
IPS inspects database traffic to identify patterns which correspond to known vulnerabilities.
(pls report back to DB venders and get patched if you find such)

- Threat 4 - Platform Vulnerabilities
Vulnerabilities in underlying operating systems (Windows 2000, UNIX, etc.) and additional services installed on a database server may lead to unauthorized access, data corruption, or denial of service.
> Prevension: Software Updates and Intrusion Prevention

- Threat 5 - SQL Injection
When a perpetrator inserts (or injects) unauthorized database statements into a vulnerable SQL data channel.
> prevention: Three techniques can be combined to effectively combat SQL injection: intrusion prevention (IPS), query-level access control, and event correlation.

- Threat 6 - Weak Audit Trail
Weakness may come from several aspects:
Lack of User Accountability when users access via web apps; degrading system performance; limited granularity etc.
> prevension: Increase performance; Separation of duty, audit duties should ideally be separate from both database administrators and the database server platform; Cross-platform auditing. Network-based audit appliances can help all these.

- Threat 7 - Denial of Service
Access to network applications or data is denied to intended users. Resource overload is particularly common in database environments.
> Prevension:requires protections at multiple levels. In this database-specific context, deployment of connection rate control, IPS, query access control, and response timing control are recommended.

- Threat 8 - Database Communications Protocol Vulnerabilities
> prevention: Protocol validation: parses (disassembles) database traffic and compares it to expectations. In

- Threat 9 - Weak Authentication
Stealing or otherwise obtaining login credentials by means of: brute force, social engineering, credential theft.
> prevention: strong authentication (in practice strong password); directory integration, use single set of login across enterprise.

- Threat 10 - Backup Data Exposure
> prevention: database backups should be encrypted.(addition: should apply the same security constraints as the original data)

In short, I will classify them into 4 categories:
- Bugs in access control(privilege design/assignment)
- software/hardware/network vulnerabilities and attacks
- User Accountability
- Backup data explosure

IPv4 addresses finally run out

2011-02-05T21:50:00.001-05:00

2^32, and finally all used up. We're in a new era.

Verizon's 2010 Data Breach Report

2011-01-12T22:29:00.003-05:00

Went through the Verizon 2010 Data Breach Investigation Report these days. Made a digest of things which are of interests to me:

Their classification of types of breach:
misuse, hacking, malware, social tactics, physical attacks.

Harm done by external agents far outweighs that done by insiders and partners. External breaches are largely the work of organized criminals. Overall, insiders were not responsible for a large share of compromised records but system and network administrators nabbed most of those that were. This finding is not surprising since higher privileges offer greater opportunity for abuse. In general, we find that employees are granted more privileges than they need to perform their job duties and the activities of those that do require higher privileges are usually not monitored in any real way.

Top 3 industries affected by data breach: Financial Services, Hospitality, and Retail. And the most popular data compromised are:
- Payment card data
- Personal information
- Bank account
- Authentication credentials

Malware and hacking composed of more than 95% of all compromised records. Cases involving the use of social tactics more than doubled. Physical attacks like theft, tampering, and surveillance ticked up several notches.

Malware factored into 38% of 2009 breach cases and 94% of all data lost. The most frequent malware infection vector is installation or injection by a remote attacker. This is often accomplished through SQL injection or after the attacker has root access to a system.

Malware functionality by percent of records:
- Backdoor 85%
- send data to external site/entity 81%
- capture data resident on system 84%
- system/network utilities(PsTools, Netcat) 83%
- Packet sniffer 80%

97% of the 140+ million records were compromised through customized malware. Some are simply repackaged versions of existing malware in order to avoid AV detection. More often they altered the code of existing malware or created something entirely new.

Two hacking types that resulted in the largest percent of data breach:
- Use of stolen credentials: 86%
Mostly obtained by malware. Ration 2:1 to other attacks including phishing, SQL injection.
- SQL injection: 89%
It is almost always an input validation failure. Main uses are for query data, modify data, and deliver malware.

Most used path of intrusion is web applications.

Nearly all data were breached from servers and applications. Breaches involving end-user devices nearly doubled in 2009. Much of this growth can be attributed to credential-capturing malware.

15% attacks are of high difficulty: Advanced skills, significant customization, and/or extensive resources required. And they contribute to 87% of data breach.

In 2009, targeted attacks accounted for 89% of records compromised.

In over 60% of breaches investigated in 2009, it took days or longer for the attacker to successfully compromise data, but 31% only takes minutes. More than 37% takes months to discover the compromises. An 29% also takes months to contain the compromise after it is discovered.

Third party fraud detection is still the most common way breach victims come to know of their predicament.

Event monitoring and log analysis successfully alerted only 6% of breach victims. This year that figure has dropped to 4%. The reason IDS doesn't work usually due to the poor configuration and monitoring. Actually 86% of the breaches have log evidence. Ways to study log: 1) abnormal increase in log data, 2) abnormal length of lines within logs, 3) absence of (or abnormal decrease in) log data.

Anti-forensics consist of actions taken by the attacker to remove, hide, and corrupt evidence or otherwise foil post-incident investigations. Data wiping, which includes removal and deletion, is still the most common but declined slightly. Data hiding rose by over 50%, and data corruption tripled. The use of encryption for the purposes of hiding data contributed most significantly to the increase in that technique while the most common use of data corruptions remains log tampering.

Techniques on large data analysis

2010-12-16T21:22:00.005-05:00

Requirement of analysis on large data sets are exploding nowadays. Recently I did a brief investigation on the techniques on large data analysis.

1. Hashing
- Usage:
   Fast searching, insertion, deletion. Data set often fits in memory.
- Design:
   * hash function selection for different data type
   * collison resolution
- Extension:
   d-left hashing: separate hash table into d segaments. Hash into the one with less collision. If tie choose the left one.

2. Bit map
- Usage:
   Fast searching, duplicate checking, deletion.
- Deisgn:
   Use bit array to represent data.
- Extension:
   * Bloom filter: Use multiple bits to represent one data point.

3. Bloom filter (I think it worths a dedicated entry here)
- Usage:
   Duplicate checking, Set union, data dictionary
- Deisgn:
   k hash functions mapping to the same domain. So for one data item there will be k set bits.
- Extension:
   Counting bloom filter: expand each bit into a counter, then it can support deletion.

4. Heap
- Usage:
   Top N in a large data set.
- Design:
   (do i need to write anything here?)
- Extension:
   Use a max-heap and a min-heap at the same time to maintain medium

5. Bucket
- Usage:
   Looking for the k-th element, medium, duplicate or non-duplicate elements
- Deisgn:
   * step 1: define the range of each bucket, count statistics in each bucket.
   * step 2: compute the target's index in the corresponding bucket, look for the target. May further seperating bucket if remaining data is still too big.

6. External Sorting
- Usage:
   Ordering, duplicate removing
- Design: (a sort-merge strategy)
   * In the sorting phase, chunks of data small enough to fit in main memory are read, sorted, and written out to a temporary file.
   * In the merge phase, the sorted subfiles are combined into a single larger file.

7. Inverted Index
- Usage:
   Allow fast full text searches. It is the most popular data structure used in document retrieval systems such as search engines.
- Deisgn:
   A mapping from content, such as words or numbers, to its locations in a database file, or in a document or a set of documents.

8 . Trie
- Usage:
   * Similar stucture of data, lots of duplicates.
   * Storing dictionary or implementing approximate matching algorithms
- Design
   Often use recursive definiton for prefix
- Extension
   Compression

9. Database index
(Classic, should I explain here?)

10. MapReduce
- Usage:
   Large dataset with limited data types. Distributed processing.
- Design
   * "Map" step: The master node takes the input, chops it up into smaller sub-problems, and distributes those to worker nodes.
   * "Reduce" step: The master node then takes the answers to all the sub-problems and combines them in some way to get the output .

Groupon Turns Down Google’s Takeover Bid

2010-12-04T11:28:00.004-05:00

6 billion, and Groupon says NO. I think for Google it maybe a good thing. But for Groupon, it says it's considering staying independent for an IPO. I really doubt about this 6-billion decision, and its growth estimation as an independent company, considering its eas-to-replicate model, and loose ads connection with "Local" business. I still think they should go with Facebook.

Google to buy Groupon?

2010-11-23T13:10:00.002-05:00

Words came out these days that google is going to buy Groupon. Some analysis about this basically suggests that it can integrate the Groupon ads into its search results, youtube videos etc. It's kind of a surprise to me as I had thought facebook would be the final buyer, especially as facebook is putting special effort into location based services. It also looked to me promising as Groupon is essentially a collective buying site, and if integrated with facebook's socail model, it's easy to add recommendation/reputation context on that. Maybe facebook is doing that by their own? Anyway, let's seat and see how things will be going.

privacy, privacy, cost of privacy ...

2010-11-15T01:26:00.004-05:00

In social network, privacy is always one of the major concern. Google just got another lesson for their negligent(?): Google settles Google Buzz privacy suit for $8.5 million donation

Era of Big Data

2010-11-07T01:32:00.008-04:00

Since the acquisition of DATAllegro from Microsoft in 2008, there are more big decisions this year: EMC bought GreenPlum, IBM bought Netezza, and Oracle upgrades Exadata(interestingly, this is announced by Mark Hurd who joined Oracle as the new president no more than half month). All of these highlight the incoming of the era of big data, and industry's leading companies' strong desire to expand their large data management and business analysis.
Here're several interesting links:
Big Data Means Big Sales
EMC's launches Greenplum appliance

Big data not only comes from the quick expanding web activities we are encountering everyday from facebook, twitter, amazon etc. The traditional industries are also generating huge amount of data every minute with the help of the latest technologies. Sensor and RFID technology has been widely used by giant companies such as walmart, target to help collect data and enable more intelligent supply chain management. And I also went across the following vision:
Sensor Networks Top Social Networks for Big Data

As visioned we're soon entering the Exa- and Zetta-byte age in the next couple of years. The imminent future of big data era calls for more aggressive advances in big data management and sharing, more intelligent and effective business analytics, and the security and privacy primitives associated with all of them.

Access Control

2010-11-06T18:09:00.007-04:00

After a user is authenticated and logon to a system, its access to resources on a computer or network system are controlled by access control modules.

Discretionary Access Control(DAC)
In a DAC model, a subject has complete control over the objects that it owns and the programs that it executes. Owner associates each of its objects with an access control list (ACL), containing a list of users and their level of access to this object. DAC is based on the owner's granting and revoking of privileges. Access to an resource is denied by default unless explicitly authorized. Most of today's OS are using DAC model.

The key weakness of DAC is that it suffers from Trojan horse attacks.

Mandatory Access Control(MAC)
MAC is the most strict of all levels of control. The MAC model targeted for systems in which confidentiality has the highest priority, such as military or government agencies. In a MAC enforced system, both subjects and objects will get assigned clearance levels(security labels). The administrator takes control of security label defintion and assignment. Access to objects are constrained by policies on the security clearance, which are also defined by administrator. The general access rule is no read up, no write down following the Bell-Lapadula Model, but it's also possible to exptend and define dedicate rules depending on the practical security requirements. MAC is fine-grained and can provide row or column level access control.

Often seen as the most secure access control environment, MAC also requires extra effort in pre-planning in order to be effectively and securely implemented. It also calls for continuous system management overhead to control new users, objects, and changes of security label defintions.

Oracle 9i has implemented label security to meet the MAC requirements and provide row level access control, and hierarchy labels are coded as numeric values. DB2 also provides LBAC to provide MAC for both row and column. The security label is composed of one or more security label components of three types: arrays(hierarchy), sets and trees.

Role Based Access Control (RBAC)
In a RBAC system, user also doesn't have discretionary access to objects. Instead, administrator create roles with a collection of permissions for different job functions or responsibilities. Each user will be assigned to one or more roles, and delegated all the privileges associated with that role. RBAC greatly simplifies the management of individual user rights and authorizations.

Many database systems have some implementation of RBAC, including Teradata, Oracle, DB2, SQL Server.

Shared Nothing Architecture

2010-11-04T17:46:00.006-04:00

Shared nothing system greatly reduces the resource contention for memory, locks, or processors. As pointed out by DeWitt et al., among the three widely used approaches, shared memory is the least scallable, shared disk medium, and shared nothing is most scalable. A shared nothing system can scale almost linearly and infinitely, simply by adding more inexpensive nodes. Shared nothing is now prevalent in the Data Warehousing space due to its potential for scaling.

As one of the earliest implementation, in teradata, each AMP virtual processor(vproc) manages its own dedicated portition of the system's disk space(vdisk, which can be multiple disk array ranks). Rows are distributed to the AMPs according to the hash of the primary index(PI). For NoPI table supported from TD 13.0, it either hashes on the Query ID for a row, or it uses a different algorithm to assign the row to its home AMP. The unconditional parallelism and linearly expandability makes its leading position in enterprise data warehousing.

Nowadays the shared nothing architecture is adopted by most high performance scalable DBMSs, including Teradata, Netezza, Greenplum, DB2 and Vertica. It is also used by most of the high-end e-commerce platforms, inclusing Amazon, Yahoo, Google, and Facebook.

In DB2 UDB Enterprise-Extended Edition (EEE), partition key is chosen as one or more columns and hash of the partition key determines which node/node group a row should be sent to.

Oracle is a shared-disk approach. In Oracle shared nothing is at logical level. Once the degree of parallelism is chosen as a power of 2, number of partitions are decided and partitions are generated by the range - hash partitioning.

Cons: Shared Nothing Architectures takes longer to respond to queries that involve joins over large data sets from different partitions. For example, in Teradata OLTP is not efficient, CPU cycles are distributed to several AMPs and PE, PEs may get easily congested by massive OLTP requests.

Always On - Aster Data example

2010-11-02T00:17:00.007-04:00

on June 29th, 2010, Google's Adwords stopped serving Ads sometime around 1:40pm PST and lasted for about 3 hours. The estimated cost is about $7.8 million. For Amazon or ebay, even some shoppers may come back later, they still lose impulse buyers, which counts for about millions per hour.

Currently zero downtime practices have been widely deployed for data migration. But for database/data warehouse, it is still a challenging problem. In general the system downtime can be classified as planned and unplanned. As 24x7 availability is becoming more and more critical for Data warehouse systems, it is expected that system is always on during the planned or unplanned downtime.

As claimed by Aster Data, they built solutions upon the Recovery-Oriented Computing to achieve this goal. The basic functionalities include:

- In-cluster replication and transparent fail-over
Data replicas are placed across the cluster, and server failure are transparently transferred to replicas within the cluster.

- Self diagnostics
If permanent failure, creating new replicas on existing or new servers without downtime. If transient failure, resync after the server recovers.

- Network aggregation
Multiple network hardware to provide parallelism and redundancy.

- Separation of duty
Dedicated servers for loading/exporting data, and backup/restore.

- Workload prediction
Policy-driven tools to manage priority of workloads and dynamically assign resources.

Column Store DBMS

2010-10-31T01:34:00.003-04:00

By its name, contents are stored by column. Sybase maybe the earliest commercial products implementing column store. Vertica and its academic precursor C-store are now a mature commercial product.

Benefits and challenges:
- Query optimization: when queries involving adding a new column for all rows, aggregating along only a few columns, column store will be much more effective.It is especially advantagerous for data warehourse, as queries are often on some specific dimentions of the data.
- High compression is possible as columns of homogeneous datatype are stored together.

In general, OLTP is more row-oriented, while OLAP is more column-oriented. Combined row and column store provides exceptional benefits and challanges in the above mentioned areas. The system level design also give more oppotunities in parallelism and failure recovery, if we take different store mechanism in the RAID or fallback protection. Aster Data, another successful startup company, claimed it has the hybrid row and column store architecture.