You are reading the article What Is Data Mart In Data Warehouse? Types & Example updated in December 2023 on the website Cancandonuts.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested January 2024 What Is Data Mart In Data Warehouse? Types & Example
What is Data Mart?A Data Mart is focused on a single functional area of an organization and contains a subset of data stored in a Data Warehouse. A Data Mart is a condensed version of Data Warehouse and is designed for use by a specific department, unit or set of users in an organization. E.g., Marketing, Sales, HR or finance. It is often controlled by a single department in an organization.
Data Mart usually draws data from only a few sources compared to a Data warehouse. Data marts are small in size and are more flexible compared to a Datawarehouse.
In this tutorial, you will learn-
Why do we need Data Mart?
Data Mart helps to enhance user’s response time due to reduction in volume of data
It provides easy access to frequently requested data.
Data mart are simpler to implement when compared to corporate Datawarehouse. At the same time, the cost of implementing Data Mart is certainly lower compared with implementing a full data warehouse.
Compared to Data Warehouse, a datamart is agile. In case of change in model, datamart can be built quicker due to a smaller size.
A Datamart is defined by a single Subject Matter Expert. On the contrary data warehouse is defined by interdisciplinary SME from a variety of domains. Hence, Data mart is more open to change compared to Datawarehouse.
Data is partitioned and allows very granular access control privileges.
Data can be segmented and stored on different hardware/software platforms.
Types of Data MartThere are three main types of data mart:
Dependent: Dependent data marts are created by drawing data directly from operational, external or both sources.
Independent: Independent data mart is created without the use of a central data warehouse.
Hybrid: This type of data marts can take data from data warehouses or operational systems.
Dependent Data MartA dependent data mart allows sourcing organization’s data from a single Data Warehouse. It is one of the data mart example which offers the benefit of centralization. If you need to develop one or more physical data marts, then you need to configure them as dependent data marts.
Dependent Data Mart in data warehouse can be built in two different ways. Either where a user can access both the data mart and data warehouse, depending on need, or where access is limited only to the data mart. The second approach is not optimal as it produces sometimes referred to as a data junkyard. In the data junkyard, all data begins with a common source, but they are scrapped, and mostly junked.
Dependent Data Mart
Independent Data MartAn independent data mart is created without the use of central Data warehouse. This kind of Data Mart is an ideal option for smaller groups within an organization.
An independent data mart has neither a relationship with the enterprise data warehouse nor with any other data mart. In Independent data mart, the data is input separately, and its analyses are also performed autonomously.
Implementation of independent data marts is antithetical to the motivation for building a data warehouse. First of all, you need a consistent, centralized store of enterprise data which can be analyzed by multiple users with different interests who want widely varying information.
Independent Data Mart
Hybrid Data Mart:A hybrid data mart combines input from sources apart from Data warehouse. This could be helpful when you want ad-hoc integration, like after a new group or product is added to the organization.
It is the best data mart example suited for multiple database environments and fast implementation turnaround for any organization. It also requires least data cleansing effort. Hybrid Data mart also supports large storage structures, and it is best suited for flexible for smaller data-centric applications.
Hybrid Data Mart
Steps in Implementing a DatamartImplementing a Data Mart is a rewarding but complex procedure. Here are the detailed steps to implement a Data Mart:
DesigningDesigning is the first phase of Data Mart implementation. It covers all the tasks between initiating the request for a data mart to gathering information about the requirements. Finally, we create the logical and physical Data Mart design.
The design step involves the following tasks:
Gathering the business & technical requirements and Identifying data sources.
Selecting the appropriate subset of data.
Designing the logical and physical structure of the data mart.
Data could be partitioned based on following criteria:
Date
Business or Functional Unit
Geography
Any combination of above
What Products and Technologies Do You Need?
A simple pen and paper would suffice. Though tools that help you create UML or ER diagram would also append meta data into your logical and physical designs.
ConstructingThis is the second phase of implementation. It involves creating the physical database and the logical structures.
This step involves the following tasks:
Implementing the physical database designed in the earlier phase. For instance, database schema objects like table, indexes, views, etc. are created.
What Products and Technologies Do You Need?
You need a relational database management system to construct a data mart. RDBMS have several features that are required for the success of a Data Mart.
Storage management: An RDBMS stores and manages the data to create, add, and delete data.
Fast data access: With a SQL query you can easily access data based on certain conditions/filters.
Data protection: The RDBMS system also offers a way to recover from system failures such as power failures. It also allows restoring data from these backups incase of the disk fails.
Multiuser support: The data management system offers concurrent access, the ability for multiple users to access and modify data without interfering or overwriting changes made by another user.
Security: The RDMS system also provides a way to regulate access by users to objects and certain types of operations.
In the third phase, data in populated in the data mart.
The populating step involves the following tasks:
Source data to target data Mapping
Extraction of source data
Cleaning and transformation operations on the data
Loading data into the data mart
Creating and storing metadata
What Products and Technologies Do You Need?
You accomplish these population tasks using an ETL (Extract Transform Load) Tool. This tool allows you to look at the data sources, perform source-to-target mapping, extract the data, transform, cleanse it, and load it back into the data mart.
In the process, the tool also creates some metadata relating to things like where the data came from, how recent it is, what type of changes were made to the data, and what level of summarization was done.
AccessingAccessing is a fourth step which involves putting the data to use: querying the data, creating reports, charts, and publishing them. End-user submit queries to the database and display the results of the queries
The accessing step needs to perform the following tasks:
Set up a meta layer that translates database structures and objects names into business terms. This helps non-technical users to access the Data mart easily.
Set up and maintain database structures.
Set up API and interfaces if required
What Products and Technologies Do You Need?
You can access the data mart using the command line or GUI. GUI is preferred as it can easily generate graphs and is user-friendly compared to the command line.
ManagingThis is the last step of Data Mart Implementation process. This step covers management tasks such as-
Ongoing user access management.
System optimizations and fine-tuning to achieve the enhanced performance.
Adding and managing fresh data into the data mart.
Planning recovery scenarios and ensure system availability in the case when the system fails.
What Products and Technologies Do You Need?
You could use the GUI or command line for data mart management.
Best practices for Implementing Data MartsFollowing are the best practices that you need to follow while in the Data Mart Implementation process:
The source of a Data Mart should be departmentally structured
The implementation cycle of a Data Mart should be measured in short periods of time, i.e., in weeks instead of months or years.
It is important to involve all stakeholders in planning and designing phase as the data mart implementation could be complex.
Data Mart Hardware/Software, Networking and Implementation costs should be accurately budgeted in your plan
Even though if the Data mart is created on the same hardware they may need some different software to handle user queries. Additional processing power and disk storage requirements should be evaluated for fast user response
A data mart may be on a different location from the data warehouse. That’s why it is important to ensure that they have enough networking capacity to handle the Data volumes needed to transfer data to the data mart.
Implementation cost should budget the time taken for Datamart loading process. Load time increases with increase in complexity of the transformations.
Advantages
Data marts contain a subset of organization-wide data. This Data is valuable to a specific group of people in an organization.
It is cost-effective alternatives to a data warehouse, which can take high costs to build.
Data Mart allows faster access of Data.
Data Mart is easy to use as it is specifically designed for the needs of its users. Thus a data mart can accelerate business processes.
Data Marts needs less implementation time compare to Data Warehouse systems. It is faster to implement Data Mart as you only need to concentrate the only subset of the data.
It contains historical data which enables the analyst to determine data trends.
Many a times enterprises create too many disparate and unrelated data marts without much benefit. It can become a big hurdle to maintain.
Data Mart cannot provide company-wide data analysis as their data set is limited.
Summary:
Define Data Mart : A Data Mart is defined as a subset of Data Warehouse that is focused on a single functional area of an organization.
Data Mart helps to enhance user’s response time due to a reduction in the volume of data.
Three types of data mart are 1) Dependent 2) Independent 3) Hybrid
Important implementation steps of Data Mart are 1) Designing 2) Constructing 3 Populating 4) Accessing and 5)Managing
The implementation cycle of a Data Mart should be measured in short periods of time, i.e., in weeks instead of months or years.
Data mart is cost-effective alternatives to a data warehouse, which can take high costs to build.
Data Mart cannot provide company-wide data analysis as data set is limited.
You're reading What Is Data Mart In Data Warehouse? Types & Example
Business Intelligence Vs Data Warehouse
Difference Between Business Intelligence vs Data Warehouse
Hadoop, Data Science, Statistics & others
Data Warehouse (DW) is simply a consolidation of data from various sources that set a foundation for Business Intelligence, which helps in making better strategic and tactical decisions. So I can say Data Warehouses have business meaning baked into them. Database stores data from different sources in a common format, and The Warehouse is like Godown (Big Building), where many things may be stored. Still, Data- Warehouse works with intelligent algorithms like Indexing, which helps locate and retrieve easily and on the same concept.
Data Warehouse is similar to a relational database aimed at querying and analyzing the data rather than for transaction processing. It usually contains historical data derived from transaction data but can include data from various sources. Data Warehouses hold data, in fact, Tables (Tables that cover numbers such as revenue and Costs) and Dimensions (Group Facts by different attributes like region, office, or week).
I will use certain abbreviations like BI for Business Intelligence and DW for Data Warehouse, as it’s easy to write. So far, I hope you have got enough understanding about both Business Intelligence and Data Warehouse concepts which are so commonly used in the Data Analytics Domain. These are so mistakenly used that even people in this domain are unsure what and when to use them.
Now let’s understand exactly what business intelligence is, which has created so much confusion in the Analytics industry as some people use both terms interchangeably. Lots of discussions are going on the internet.
A Business Intelligence system tells you what happened or is happening in your business; it describes the situation to you. Also, a good BI platform represents this to you in real time in a granular, accurate, and presentable form.
I will tell you why it is so intelligent; using Data is simple. Data is accumulated over a significant amount of time from several disparate sources.
But now, a fundamental question arises about where this data is. This data is stored in the Data Warehouse (DDS, Cubes). And BI systems use Data Warehouse data and let you apply chosen metrics to potentially massive, unstructured data sets and cover querying, data mining, online analytical processing (OLAP), reporting, and business performance monitoring predictive and prescriptive analytics.
So now let’s compare Business Intelligence and Data Warehouse to better understand by comparing.
Head-to-Head Comparison Between Business Intelligence vs Data Warehouse (Infographics)Below are the top 5 comparisons between Business Intelligence vs Data Warehouse:
Key Differences Between Business Intelligence vs Data WarehouseFollowing are the differences between Business Intelligence vs Data Warehouse:
BI means finding insights that portray a business’s current picture (How and What) by leveraging data from the Data Warehouse (DW).
BI is about accessing and exploring an organization’s data, while Data Warehouse is about gathering, transforming, and storing data.
DW outlines the actual Database creation and integration process along with Data Profiling and Business validation rules. At the same time, Business Intelligence uses tools and techniques that focus on counts, statistics, and visualization to improve business performance.
BI deals with OLAP, data visualization, data mining, and query/reporting tools. In contrast, DW deals with data acquisition, metadata management, data cleansing, data transformation, data distribution, and data recovery/backup planning.
DW teams use tools like Ab Initio Software, Amazon Redshift, Informatica, etc., while BI teams use tools like Cognos, MSBI, Oracle BI, Pentaho, QlikView, etc.
Software engineers, mainly Data Engineers, deal with DW, while top executives, Managers deal with BI.
Business Intelligence vs Data Warehouse Comparison TableBelow is the comparison table between Business Intelligence vs Data Warehouse.
Basis for Comparison Business Intelligence Data Warehouse
What it is System for deriving insights related to business. Data Storage: historical along with the current.
Source Data from Data warehouse. Data from several Data sources and applications.
Output Business reports, charts, graphs. Data, in fact, and dimension tables for upstream applications or BI tools.
Audience Top executives, Manager Data Engineers, Data Analysts, and Business Analysts.
Tools MSBI, QlikView, Cognos, etc. Ab Initio Software, Amazon Redshift, Informatica.
ConclusionSo I finally want to conclude this article as BI tools like QlikView, MSBI, and Oracle BI all access data from Data Warehouses. And let business users create more granular and presentable reports, graphs, and charts, which help top executives to make more effective business decisions in different functional areas like finance, supply chain, human resources, sales & marketing, and customer service.
Recommended ArticlesThis is a guide to Business Intelligence vs Data Warehouse. Here we have discussed head-to-head comparison, key differences, and a comparison table. You may also look at the following articles to learn more –
The Intranet Data Warehouse: A Cultural Revolution
“The key characteristic of predators is that they do not think like the established competitors.”
Information Advantage’s Rick Tanler
What is the information culture of your enterprise? Is now the time to change your information culture? Who is responsible for changing the information culture? These are the important questions as organizations begin to make data warehouse content available on their intranet.
When information services managers talk about data warehousing, the focus is on the management of alpha-numeric content stored in databases and the application software for user-initiated reporting and analysis of this data. But consider this: The World Wide Web is the largest data warehouse on earth. It informs millions of users every day. The Internet has been successful, in part, because the emphasis has shifted from personal computing to global information sharing—from delivering application features to delivering content. This is the most valuable lesson of the Internet and it alters the information culture of the Enterprise.
The adoption of Internet technologies, to build secure Intra/Extranets, provides organizations with a remarkably efficient and highly capable means of informing everyone—employees, affiliates, customers, and prospects. Arguably, the most valuable sources of internal business intelligence are contained in the data warehouses.
The discussions of three technologies—data warehousing, online analytic processing (OLAP) and intranets—have become intertwined. The result is a powerful blending of technologies that offers new ways to support the information needs of users. How will your organization seize the opportunity? The answer may lie in changing the information culture of your enterprise.
Virtually every company has an information culture. It is reflected in the priorities established and the methods used to manage, distribute, and use information. The most common part of the information culture is the spectator. The spectator views information as being essential to monitoring every change that impacts business operation from a transaction perspective. Spectators describe their data warehouses as ‘the source of the data for reporting and analysis.’ To them, the focus of information services is on meeting users’ requirements for reports. Every organization starts by being a spectator.
The more interesting segment of the information culture occurs in the organization that recognizes that substantial rewards go to the best competitors, not to spectators. These “competitors” view the data warehouse as a catalyst for changing business-management processes in order to become a market leader. The competitor talks about information in terms of how it supports the decision process. Rather than being better at observing what happened, the competitor is intent on making things happen. He recognizes that access to information initiates change.
In The Intranet Data Warehouse (John Wiley & Sons), I differentiated these two cultures—spectators and competitors—by suggesting that one focused on meeting users’ decision support requirements while the other focused on the requirements to improve decision implementation. This separation worked until I began to recognize that there is a third kind of information culture: predators.
Predators occur in an organization that changes the fundamental rules of competition. The predator establishes a new market in its attack on existing markets. chúng tôi is a predator. By creating a virtual bookstore, Amazon found a way to avoid the capital intensive “brick and mortar” retail business model. The key characteristic of predators is that they do not think like the established competitors.
10 Best Data Warehouse Books You Should Read In 2023
Books To Read About Data Warehouse
The top 10 derivative books cover a broad spectrum of topics related to data warehousing, including data modeling, ETL processes, project management, and implementation with various technologies. They are here to guide you so you can invest your valuable rupees in books worth your time.
Start Your Free Data Science Course
Hadoop, Data Science, Statistics & others
10 Different Data Warehouse Books# Books Author Published Rating
1. The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, 3rd Edition Ralph Kimball 2013 Amazon: 4.7
2. Agile Data Warehouse Design: Collaborative Dimensional Modeling, from Whiteboard to Star Schema Lawrence Corr, Jim Stagnitto
2011 Amazon:4.6
3. Database Systems: Introduction to Databases and Data Warehouses Nenad Jukic, Susan Vrbsky, Svetlozar Nestorov
2023 Amazon: 4.5
4. Building a Scalable Data Warehouse with Data Vault 2.0 Daniel Linstedt, Michael Olschimke 2023 Amazon:4.4
5. The Kimball Group Reader: Relentlessly Practical for Data Warehousing and Business Intelligence Remastered Collection
2010 Amazon:4.6
6. Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing (Agile Software Development Series) Ken Collier
2011
Amazon: 4.3
7. Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale Tom White 2023
Amazon 4.5
8. Building a Data Warehouse: With Examples in SQL Server (Expert’s voice) Vincent Rainardi 2008
9. Big Data: Principles and best practices of scalable real-time data systems Nathan Marz and James Warren
2012
10. Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems Martin Kleppmann 2023 Amazon: 4.8
Let us look at the Data Warehouse books and see which one best suits your needs:-
1. The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, 3rd EditionAuthors: Ralph Kimball and Margy Ross
Get this Book here
Book Review
A great addition to a businessman’s most used tools for knowing about BI/DW. The Book does well to promise to be helpful to both students and professionals alike. This reference book is necessary if you are thinking about repetitive analogies that will permanently reside in your brain.
Key Takeaway from that Book
The reading material includes well-made case studies from retail sales, financial services, etc., to derive the meaning of the Book’s concept in real life.
It contains value chain introduction, procurement transactions, bus matrix, order management, accounting, etc.
Authors: Lawrence Corr and Jim Stagnitto
Get this Book here
Book Review
The authors have done an excellent job presenting the topics of the agile manifesto with the BEAM approach that is perfect for people who want another feather in their cap through data warehousing. The author uses visual representation tools such as diagrams and little characters to make the text seem less daunting. The readers get a glimpse of modeling a data warehouse from working on the whiteboard to star schema and various other topics throughout the Book.
Key Takeaway from that Book
The topics of modeling business events, modeling star schemas, and design patterns for high-performance fact tables are explained in great detail for users to understand completely.
A unique feature of the Book is that we get summaries at the end of each chapter to glance back at the topics just read.
3. Database Systems: Introduction to Databases and Data WarehousesAuthors: Nenad Jukic, Susan Vrbsky, and Svetlozar Nestorov
Get this Book here
This Book has a good command over operational and analytical database systems and creating ERDplus, relational and dimensional models. This Book is an introductory guide to use in university classes.
Key Takeaway from that Book
Ideal for students looking to start as novices in data warehousing who want to practice with many exercises and practice sheets.
The excellently portrayed topics are ERD diagrams and relational entity models.
4. Building a Scalable Data Warehouse with Data Vault 2.0Author: Daniel Linstedt and Michael Olschimke
Get this Book here
Book Review
Welcoming you to a world of data vaults in data warehousing, this Book takes you on a wild ride touching the concepts of SQL Server Integration Services, data quality services, and master data services. Many topics are discussed in great detail with suitable diagrams and required references at the end.
Key Takeaway from that Book
This Book is for anyone wanting to dive into the data vault though it was a little outdated in 2023.
5. The Kimball Group Reader: Relentlessly Practical for Data Warehousing and Business Intelligence Remastered CollectionAuthors: Ralph Kimball, Margy Ross, Bob Becker, Joy Mundy, and Warren Thornthwaite
Get this Book here
Book Review
The authors have incorporated their year’s knowledge of data warehousing in the final series of the anthology, which is more comprehensive and expanded from the previous editions. The Book requires a prerequisite knowledge of the basics of the data warehouse discussed.
Key Takeaway from that Book
The Book covers 65 new articles and distinguishable features which were not there in the previous editions.
The book website has content with design tips available and is easily accessible for the general public to consume.
Outlines topics such as fact tables, ETL subsystems, data quality, BI application best practices, etc., briefly.
6. Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing (Agile Software Development Series)Author: Ken Collier
Get this Book here
Book Review
The Book is easy to follow and precise, with information about agile analytics. The Book contains topics such as filibuster analytics and some truths about agile analytics, which are easy to understand.
The key takeaway from that Book
The topics of management methods and technical methods are heavily focused On Agile analytics.
The primary topics of these books are modeling business events, modeling business dimensions and facts, and measures and KPIs.
The Book is a complete meal for someone hungry to know much about agile for evolving excellent design, version control for data warehousing, and user stories for BI systems.
7. Hadoop: The Definitive Guide: Storage and Analysis at Internet ScaleAuthor: Tom White
Get this Book here
Book review
Tom White does an excellent job at capturing the core of Hadoop fair and square without meandering about nonsense. This book is a guide for everyone new to Hadoop open-source software. This Book is for anyone with some prerequisite knowledge of the open source.
Key Takeaway from that Book
The Book covers topics like how MapReduce works and administering Hadoop, Avro, parquet, and Hbase.
The Book contains ample code and theory to get you invested in Hadoop.
8. Building a Data Warehouse: With Examples in SQL Server (Expert’s Voice)Author: Vincent Rainardi
Book Review
A thorough guide to building a data warehouse simply yet eloquently. Promises an exhaustive list of data extraction, metadata, the architecture of data warehousing, etc. Experienced developers, managers working in the data warehouse field, and admins of DW are the main target this Book has been aimed for.
Key Takeaway from that Book
The Book thoroughly discusses building reports, populating the data warehouse, and data modeling.
The Book is built around a cluster of real-world cases with which the author had a chance to interact.
A must-have if you want access to practical implementation in a DW book using Microsoft SQL.
9. Big Data: Principles and best practices of Scalable real-time data SystemsAuthor: Nathan Marz
Get this Book here
Book Review
It is essentially a guide for people interested to know how technologies fit together in big data and how to choose the ideal tech stack for a specific problem. Recommended for anyone who wants to skim over the surface of the data warehouse.
Key Takeaway from that Book
An organized accumulated read-on data model for big data, serving layers, and stream processing.
Lambda architecture is the major attraction of this Book, and the code blocks of teaching Hadoop are written in Java, which can confuse some readers.
10. Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable SystemsAuthor: Martin Kleppmann
Get this Book here
Book Review
A contemporary guide to unlocking concepts of modern networked applications that will stay in your mind long after you finish the Book. A primary focus is on the functionality of each model rather than the practical implementation.
Key Takeaway from that Book
The Book is divided into three parts: foundations of data systems, distributed data, and derived data.
The illustrations make the chapters enjoyable, while the text is helpful with accurate code snippets.
Batch processing, trouble with distributed systems, data models and query languages, and replication are some topics that are heavily focused on.
Recommended ArticlesOur Top 10 Data Warehouse books compilation aims to be helpful to you. For more such Data Warehouse books, EDUCBA recommends the following,
What Is Data Storage And How Is It Used?
As modern companies rely on data, establishing dependable, effective solutions for maintaining that data is a top task for each organization. The complexity of information storage technologies increases exponentially with the growth of data. From physical hard drives to cloud computing, unravel the captivating world of data storage and recognize its ever-evolving role in our daily lives.
What is Data Storage?Data storage, or data keeping, is storing information and making it as readily available as possible via technology designed particularly for that purpose. It constitutes a simple method of storing data in digital form on computer devices, and keeping data on hand makes many digital processes more effective.
Storage devices may use electromagnetic, optical, or other media to keep the data safe and recover it when necessary. File recovery and backup procedures become simple by data storage in the case of an unforeseen computer failure or cyberattack.
While setting up data storage, every organization should consider these three factors: dependability, affordability of the storage structure and safety features.
Source: vectorStock
Also Read: Understanding the Basics of Data Warehouse and its Structure
Why Do We Need Data Storage?Innovative technologies like data analysis, the Internet of Things, and AI produce and utilize enormous amounts of data. Therefore, data storage plays a major role in the growth of any organization now more than ever. Some of the benefits of data storage are as follows:
It is simple to gather large amounts of records for a longer time using electronic data storage.
Making duplicates of stored data makes it simple to back it up, enabling file loss or corruption recoverable more quickly and easily.
With today’s cutting-edge security technologies and capabilities, plenty of techniques exist to store and safeguard particularly sensitive data digitally safely.
Every authorized individual has access to centralized stored data, which can be viewed and shared between teams whenever they collaborate.
Digital data can be more easily categorized and organized, and the process can be accessed using a desktop computer or similar connected device.
Digital data storage is faster than producing files that must be kept in file cabinets by printing out hard copies of data.
Types of Data StorageThere are majorly three types of data storage as follows:
Primary StorageA computer system’s primary data storage serves as its primary storage. The primary storage is temporary memory, also referred to as cache memory. Primary memory is invariably smaller than secondary memory and comes with comparatively lesser storage. It is the only storage form readily available to the CPU, unlike RAM and ROM (Read-only Memory). The CPU always accesses primary storage-stored commands and processes them as needed. All data that is actively worked on is kept in an organized manner.
Secondary StorageA secondary storage system can keep data longer and has additional storage space. External or internal computer components include hard drives, USB drives, CDs, and other media. One computer typically accesses secondary storage through its input/output channels and transfers the needed data utilizing an intermediate space in primary storage.
Tertiary StorageIt is an extensive electronic storage system that is typically quite sluggish; hence, it stores components that are retrieved occasionally. This method often incorporates a robotic device that mounts and dismounts removable drives into computer storage units following the system requirements. It helps access extremely massive databases without the assistance of any human controllers.
Traditional Storage TechnologiesThe conventional data storage systems are as follows:
Magnetic StorageA magnetic memory, such as an HDD, comprises circular drives composed of non-magnetic components and coated using a thin film of magnetic material, where data is stored. The magnetized face of such disks go inside a rotary drive, with a read-write unit of a magnetic yoke and a magnetizing coil that spins in close range of the disks.
Optical StorageAn optical drive is a device that uses optical storage techniques for data processing functions such as read/write/access. Laser light helps in reading and storing data on an optical disk. An optical disk is a resin similar to polycarbonate, and the electronic data is maintained in tiny openings on the polycarbonate layer.
Modern Data Storage TechnologiesThe modern data storage systems are as follows:
Flash StorageFlash storage uses solid-state drives (SSDs) with flash memory for large-scale data or file archiving. It substitutes HDDs and other forms of storage. A multi-terabyte dataset can be kept “in memory” using an all-flash array, which offers read/write speeds four times faster than HDDs. Compared to HDDs, flash storage has a higher density.
Cloud StorageBy replicating the capability of physical storage devices, cloud storage enables you to save or retrieve various content types whenever you need to from a virtual setting. Any data uploaded to the cloud is kept off-site in reliable data centers, and an on-site operator or an off-site third-party service often handles it. Users can access cloud storage using a computer with an internet connection, web portal, intranet, cloud storage apps, or additional application programming interfaces (APIs).
Object StorageObject storage is a technique that manages data storage in distinct components or objects. A framework on which data analytics software can run queries on objects is known as an object store. By adopting a flat address space, object storage removes the need for the hierarchical structure that different systems need. This enables easy scaling up or down to accommodate storage workload variations and accommodate quick expansions and contractions.
Software-Defined StorageSDS is a storage system that separates the hardware and software used for storage. Unlike conventional NAS or SAN systems, SDS runs on any x86 or industry-standard system, eliminating the software’s reliance on specific hardware. Software-defined storage is a method of managing data that makes data storage resources more flexible by abstracting them from the supporting physical storage hardware.
How Does Data Storage Work?Whenever you upload digital data to a personal computer, it gets saved to a device, which stays there until it is damaged. Storage is fundamentally different from computer memory: While anyone can swiftly retrieve information from your computer’s RAM, such data is only accessible in RAM when your computer is off.
Modern computers or devices may connect to storage devices directly or via a network connection. Users give computers instructions for accessing data stored on and retrieved from various storage devices. On a basic level, data storage depends on two principles: the form it takes and the hardware that it is captured and stored on.
Data Storage Architectures and ConceptsThe distinct data storage architectures and concepts are as follows:
RAID (Redundant Array of Independent Disks)RAID is a method that uses several drives in tandem rather than just one to boost performance, provide data redundancy, or both. It is a method for securing data during a drive crash by maintaining the same data in different places on many hard drives or solid-state storage devices. It has two or more parallel-operating disks, and RAID level is how disks are arranged.
A network-attached storage (NAS) server is a specific storage platform that links to devices over a LAN. The server’s connectivity features allow for the retrieval and storage of data from many external devices, and NAS storage also offers extensive sharing capabilities. The system utilizes the features of a file-storage technology and the clustering of a redundant array of drives (RAID).
Source: Phonixnap
SAN (Storage Area Network)A storage area network (SAN) is a network-based storage that can access data at the block level. This kind of storage consists of several data storage units connected by a network. The storage format is an amalgam of NAS and DAS. The storage type transfers data across a server and storage using specific networking protocols, like Fibre Channels.
The difference between object storage and block storage are as follows:
Object StorageBlock StorageData is held in flat-file systems as different, distinctive, and recognizable components called objects.Fixed-sized blocks divide the data into sections and rearrange it when necessary.Cost-effective ExpensiveUnlimited Scalability Limited ScalabilityA single central or decentralized system that maintains data in the private, public, hybrid, or cloud.A centralized system for on-site or private cloud data storage. If the program and its data storage are located far away from one another, latencies could pose a concern.Suitable for large amounts of raw data. Large files yield the greatest chúng tôi for storing databases and data related to transactions. It performs best with compact files.
Best Practices for Data Storage ManagementSome of the best practices for data storage management are as follows:
Data Backup and RecoveryAfter you transfer your data from regular, operational systems for immediate and future storage, a reliable data backup and recovery plan will ensure it is constantly kept secure. Backup copies enable data to be recovered from an earlier date, enabling the organization to recover from unforeseen circumstances. Maintaining another copy of the data on another storage device is important for protecting against original data loss or corruption.
Data DeduplicationThere are instances where similar data is produced due to repeated operations. You can improve data management and reduce storage costs by setting up a human or automated procedure that constantly evaluates data and eliminates duplicates. Your data will remain clean and prepared for evaluation and questions.
Data CompressionData compression makes files take up less room on a hard drive and takes less time to transfer or download. The decrease in distance and time could lead to major savings in expenses. It makes it possible to transport data objects and files quickly through networks and the Internet while maximizing the use of physical storage space.
Data Security and EncryptionIt measures enable you to identify sensitive data and essential assets, and establish robust security measures that monitor and protect every stage of data sorting, thereby maximizing your data security. Encryption converts the data you store into nonsensical codes; only the owner’s key can decode it. This ensures the data won’t be used, even when unauthorized people gain access to it.
Data Storage in Mobile Devices Internal StorageThe internal memory space of the device is the internal storage. The files you maintain here are restricted to the application itself, so no matter their permissions, other applications cannot access those. Android OEMs and app developers utilize internal storage to store private data, app data, user settings, and additional system files.
External StorageAny storage not part of the device’s internal memory, including an attached SD card, is called external storage. Any app with the appropriate permissions could have access to this region, which serves as a free-for-all area. There are two kinds of external storage: SD cards, commonly called memory cards, which represent the secondary external storage, and built-in external storage, which is the primary external storage.
What’s Next? Frequently Asked QuestionsQ1. What are some examples of tertiary storage?
A. Magnetic tape, optical discs, and optical tapes are a few examples of tertiary storage. These gadgets have distinctive portable storage components and are made up of fixed drivers.
Q2. What is the most popular type of storage?
A. A conventional hard disk (HDD) is one of the most widely used media storage systems.
Q3. Where is SDS used?
A. Software-defined storage (SDS) is a technique for managing data storage that carefully divides the operations responsible for allocating resources, securing data, and managing and placing data from the hardware needed to store data.
Related
Understanding Excel Vba Data Types (Variables And Constants)
In Excel VBA, you would often be required to use variables and constants.
When working with VBA, a variable is a location in your computer’s memory where you can store data. The type of data you can store in a variable would depend on the data type of the variable.
For example, if you want to store integers in a variable, your data type would be ‘Integer’ and if you want to store text then your data type would be ‘String’.
More on data types later in this tutorial.
While a variable’s value changes when the code is in progress, a constant holds a value that never changes. As a good coding practice, you should define the data type of both – variable and constant.
When you code in VBA, you would need variables that you can use to hold a value.
The benefit of using a variable is that you can change the value of the variable within the code and continue to use it in the code.
For example, below is a code that adds the first 10 positive numbers and then displays the result in a message box:
Sub AddFirstTenNumbers() Dim Var As Integer Dim i As Integer Dim k as Integer For i = 1 To 10 k = k + i Next i MsgBox k End SubThere are three variables in the above code – Var, i, and k.
The above code uses a For Next loop where all these three variables are changed as the loops are completed.
The usefulness of a variable lies in the fact that it can be changed while your code is in progress.
Below are some rules to keep in mind when naming the variables in VBA:
You can use alphabets, numbers, and punctuations, but the first number must be an alphabet.
You can not use space or period in the variable name. However, you can use an underscore character to make the variable names more readable (such as Interest_Rate)
You can not use special characters (#, $, %, &, or !) in variable names
VBA doesn’t distinguish between the case in the variable name. So ‘InterestRate’ and ‘interestrate’ are the same for VBA. You can use mixed case to make the variables more readable.
VBA has some reserved names that you can use for a variable name. For example, you can not use the word ‘Next’ as a variable name, as it’s a reserved name for For Next loop.
Your variable name can be up to 254 characters long.
To make the best use of variables, it’s a good practice to specify the data type of the variable.
The data type you assign to a variable will be dependent on the type of data you want that variable to hold.
Below is a table that shows all the available data types you can use in Excel VBA:
Data Type Bytes Used Range of Values
Byte 1 byte 0 to 255
Boolean 2 bytes True or False
Integer 2 bytes -32,768 to 32,767
Long (long integer) 4 bytes -2,147,483,648 to 2,147,483,647
Single 4 bytes -3.402823E38 to -1.401298E-45 for negative values; 1.401298E-45 to 3.402823E38 for positive values
Double 8 bytes -1.79769313486231E308 to-4.94065645841247E-324 for negative values; 4.94065645841247E-324 to 1.79769313486232E308 for positive values
Currency 8 bytes -922,337,203,685,477.5808 to 922,337,203,685,477.5807
Decimal 14 bytes +/-79,228,162,514,264,337,593,543,950,335 with no decimal point;+/-7.9228162514264337593543950335 with 28 places to the right of the decimal
Date 8 bytes January 1, 100 to December 31, 9999
Object 4 bytes Any Object reference
String (variable-length) 10 bytes + string length 0 to approximately 2 billion
String (fixed-length) Length of string 1 to approximately 65,400
Variant (with numbers) 16 bytes Any numeric value up to the range of a Double
Variant (with characters) 22 bytes + string length Same range as for variable-length String
User-defined Varies The range of each element is the same as the range of its data type.
When you specify a data type for a variable in your code, it tells VBA to how to store this variable and how much space to allocate for it.
For example, if you need to use a variable that is meant to hold the month number, you can use the BYTE data type (which can accommodate values from 0 to 255). Since the month number is not going to be above 12, this will work fine and also reserve less memory for this variable.
On the contrary, if you need a variable to store the row numbers in Excel, you need to use a data type that can accommodate a number up to 1048756. So it’s best to use the Long data type.
As a good coding practice, you should declare the data type of variables (or constants) when writing the code. Doing this makes sure that VBA allocates only the specified memory to the variable and this can make your code run faster.
Below is an example where I have declared different data types to different variables:
Sub DeclaringVariables() Dim X As Integer Dim Email As String Dim FirstName As String Dim RowCount As Long Dim TodayDate As Date End SubTo declare a variable data type, you need to use the DIM statement (which is short for Dimension).
In ‘Dim X as Integer‘, I have declared the variable X as Integer data type.
Now when I use it in my code, VBA would know that X can hold only integer data type.
If I try to assign a value to it which is not an integer, I will get an error (as shown below):
Note: You can also choose to not declare the data type, in which case, VBA automatically considers the variable of the variant data type. A variant data type can accommodate any data type. While this may seem convenient, it’s not a best practice to use variant data type. It tends to take up more memory and can make your VBA code run slower.
While you can code without ever declaring variables, it’s a good practice to do this.
Apart from saving memory and making your code more efficient, declaring variables has another major benefit – it helps trap errors caused by misspelled variable names.
To make sure you’re forced to declare variables, add the following line to the top of your module.
Option ExplicitWhen you add ‘Option Explicit’, you will be required to declare all the variables before running the code. If there is any variable that has not been declared, VBA would show an error.
There is a huge benefit in using Option Explicit.
Sometimes, you may end up making a typing error and enter a variable name which is incorrect.
Normally, there is no way for VBA to know whether it’s a mistake or is intentional. However, when you use ‘Option Explicit’, VBA would see the misspelled variable name as a new variable that has not been declared and will show you an error. This will help you identify these misspelled variable names, which can be quite hard to spot in a long code.
Below is an example where using ‘Option Explicit’ identifies the error (which couldn’t have been trapped had I not used ‘Option Explicit’)
Sub CommissionCalc() Dim CommissionRate As Double CommissionRate = 0.1 Else CommissionRtae = 0.05 End If MsgBox "Total Commission: " & Range("A1").Value * CommissionRate End SubNote that I have misspelled the word ‘CommissionRate’ once in this code.
If I don’t use Option Explicit, this code would run and give me the wrong total commission value (in case the value in cell A1 is less than 10000).
But if I use Option Explicit at the top of the module, it will not let me run this code before I either correct the misspelled word or declare it as another variable. It will show an error as shown below:
While you can insert the line ‘Option Explicit’ every time you code, here are the steps to make it appear by default:
Check the option – “Require Variable Declaration”.
Once you have enabled this option, whenever you open a new module, VBA would automatically add the line ‘Option Explicit’ to it.
Note: This option will only impact any module you create after this option is enabled. All existing modules are not affected.
So far, we have seen how to declare a variable and assign data types to it.
In this section, I will cover the scope of variables and how you can declare a variable to be used in a subroutine only, in an entire module or in all the modules.
The scope of a variable determines where can the variable be used in VBA,
There are three ways to scope a variable in Excel VBA:
Within a single subroutine (Local variables)
Within a module (Module-level variables)
In all modules (Public variables)
Let’s look at each of these in detail.
When you declare a variable within a subroutine/procedure, then that variable is available only for that subroutine.
You can not use it in other subroutines in the module.
As soon as the subroutine ends, the variable gets deleted and the memory used by it is freed.
In the below example, the variables are declared within the subroutine and would be deleted when this subroutine ends.
When you want a variable to be available for all the procedures in a module, you need to declare it at the top of the module (and not in any subroutine).
Once you declare it at the top of the module, you can use that variable in all the procedures in that module.
In the above example, the variable ‘i’ is declared at the top of the module and is available to be used by all the modules.
Note that when the subroutine ends, the module level variables are not deleted (it retains its value).
Below is an example, where I have two codes. When I run the first procedure and then run the second one, the value of ‘i’ becomes 30 (as it carries the value of 10 from the first procedure)
If you want a variable to be available in all the procedure in the workbook, you need to declare it with the Public keyword (instead of DIM).
The below line of code at the top of the module would make the variable ‘CommissionRate’ available in all the modules in the workbook.
Public CommissionRate As DoubleYou can insert the variable declaration (using the Public keyword), in any of the modules (at the top before any procedure).
When you work with local variables, as soon as the procedure ends, the variable would lose its value and would be deleted from VBA’s memory.
In case you want the variable to retain the value, you need to use the Static keyword.
Let me first show you what happens in a normal case.
In the below code, when I run the procedure multiple times, it will show the value 10 everytime.
Sub Procedure1() Dim i As Integer i = i + 10 MsgBox i End SubNow if I use the Static keyword instead of DIM, and run the procedure multiple times, it will keep on showing values in increments of 10. This happens as the variable ‘i’ retains its value and uses it in the calculation.
Sub Procedure1() Static i As Integer i = i + 10 MsgBox i End SubWhile variables can change during the code execution, if you want to have fixed values, you can use constants.
A constant allows you to assign a value to a named string that you can use in your code.
The benefit of using a constant is that it makes it easy to write and comprehend code, and also allows you to control all the fixed values from one place.
For example, if you are calculating commissions and the commission rate is 10%, you can create a constant (CommissionRate) and assign the value 0.1 to it.
In future, if the commission rate changes, you just need to make the change at one place instead of manually changing it in the code everywhere.
Below is a code example where I have assigned a value to the constant:
Sub CalculateCommission() Dim CommissionValue As Double Const CommissionRate As Double = 0.1 CommissionValue = Range("A1") * CommissionRate MsgBox CommissionValue End SubThe following line is used to declare the constant:
Const CommissionRate As Double = 0.1When declaring constants, you need to start with the keyword ‘Const‘, followed by the name of the constant.
Note that I have specified the data type of the constant as Double in this example. Again, it’s a good practice to specify the data type to make your code run faster and be more efficient.
If you don’t declare the data type, it would be considered as a variant data type.
Just like variables, constants can also have scope based on where and how these are declared:
Within a single subroutine (Local constants): These are available in the subroutine/procedure in which these are declared. As the procedure ends, these constants are deleted from the system’s memory.
Within a module (Module-level constants): These are declared at the top of the module (before any procedure). These are available for all the procedures in the module.
In all modules (Public constants): These are declared using the ‘Public’ keyword, at the top of any module (before any procedure). These are available to all the procedures in all the modules.
You May Also Like the Following VBA Tutorials:
Update the detailed information about What Is Data Mart In Data Warehouse? Types & Example on the Cancandonuts.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!