knowledge
Step 1: Getting Your Data In Order
Caitlin Aronin
If you’re planning a website redesign, replatform, or migration, there’s one step that will make or break the entire project:
Getting your data in order.
Whether you’re moving to a new ecommerce platform or updating your existing stack, how clean and structured your data is determines how smooth the project will go, how long it will take, and how successful your new site will be at launch.
This guide is built from real-world experience across dozens of builds. It combines insights from our Global Delivery Manager, Kaushal Shah — who sees data issues impact timelines firsthand — and Bhupendra Jadeja, our development team squad lead, who deals with the technical consequences of messy, inconsistent, or incomplete data.
If you’re starting a project soon (or thinking about one), this is the guide you need to read.
Why Data Is the First Step — Not an Afterthought
“Project delays due to data issues happen more often than you imagine,” Kaushal shared.
Data goes far beyond product names and images. Ecommerce requires significantly more data than ERP or POS systems that power offline sales. “For merchants who are starting fresh with their ecommerce channel, it’s important to understand that the data served on ecommerce will be different than what may have traditionally been used for other channels.”
Why it matters:
For most ecommerce teams, data required includes:
Product information (attributes, images, pricing, dimensions)
Inventory & stock status
Categories
Customers & segmentation
Orders & history
Sales and catalog rules
Custom attributes
CMS content
Data from ERP, PIM, CRM, POS, or OMS systems
If even one of these areas is incomplete or inconsistent, it can block development, break features on the new site, or require costly rework.
For B2B merchants, pricing data is often the biggest challenge. “With customer-specific pricing, price books can be complex and while these work fine in backend systems, it’s often difficult to replicate in the ecommerce channel,” said Kaushal.
Signs Your Data Isn’t Ready Yet
Kaushal shared two early red flags that we look out for when starting a project:
1. Difficulty providing sample data
If exporting a small batch of products, customers, or orders is difficult, that’s a sign deeper cleanup is needed.
2. Unclear “source of truth” across systems
“If there is confusion when creating an ecosystem map, that’s a red flag we may not receive the data in time.”
If your team isn’t sure which system is the source for pricing, which holds the authoritative product data, or how categories map between tools, data prep becomes a major project in itself.
These warning signs don’t mean you shouldn’t proceed, they mean you should prioritize data readiness early, before design or development begins.
How “Looks Fine to Me” Data Breaks During a Build
Our developers routinely encounter data that looks perfectly normal in a spreadsheet, but immediately breaks scripts, import processes, or APIs.
Common silent killers:
CSV issues
extra spaces or line breaks
text fields with commas not wrapped in quotes
inconsistent delimiters
misaligned columns after a single formatting mistake
Example:
Without quotes, this becomes four columns instead of three. The whole import fails.
API issues
smart quotes, emojis, or non-UTF-8 characters
prices like “1,999.00” instead of 1999.00
inconsistent date formats (DD/MM/YYYY vs MM/DD/YYYY)
Boolean fields populated with “Yes/No” instead of true/false
missing required identifiers like SKU or store ID
Bhupendra described this scenario simply: “It looks normal in a spreadsheet, but automation fails immediately.”
How Data Issues Break Your New Site’s Front-End
Even if the import works, messy data often breaks customer-facing features. Here are examples straight from our team:
Filters showing no products
↳ Cause: inconsistent attribute values (e.g., “Red” vs “red”).
Products missing from search results
↳ Cause: product enabled globally but disabled at a store-view level.
Variants not functioning
↳ Cause: missing parent-child relationships or mismatched attributes.
Products marked "Out of Stock" incorrectly
↳ Cause: mismatched SKU references or missing inventory data.
Shipping rates failing or calculating incorrectly
↳ Cause: missing or inconsistent product attributes like weight and dimensions.
As Kaushal explained, “shipping carriers require accurate dimensional data to return the correct rate.” If these values are missing or formatted incorrectly, the system can’t calculate shipping costs, which leads to checkout errors, inaccurate rates, or abandoned carts.
Broken images or stretched product tiles
↳ Cause: inconsistent image aspect ratios or incorrect media types.
Sorting behaving unpredictably
↳ Cause: price or date fields stored as text instead of numbers.
“Product grouping issues lead to variants appearing with identical images, making it hard for customers to select the right option,” shared Kaushal.
Small data problems can become real UX problems fast.
The Extra Work Developers Have to Do When Data Isn’t Ready
Messy data doesn’t just cause bugs; it creates entire new tasks that weren’t in scope.
Our dev team listed the most common additions:
writing data-cleaning scripts
normalizing text, numbers, or date fields
adding defensive code to avoid null errors
debugging issues tied to specific records
repairing mismatched IDs, SKUs, or relationships
reworking abandoned carts, order histories, or customer imports
resolving platform limit violations (one store had 800+ sales rules)
That last example required core-level customizations just to stabilize performance.
What Clean, Ready Data Actually Looks Like
Everyone on the team agreed on this part.
Clean data:
is complete
is consistent
uses correct formats
follows the platform’s expectations
is free of duplicates
maps cleanly across systems
imports without warnings or errors
“Clean data = faster development + faster testing + faster deployment + happier team,” shared Bhupendra.
Framework for Getting Data Ready Before a Project
Here’s the method we use with our clients to get their data in shape before development begins.
1. Perform a Gap Analysis
Compare existing data to the fields and formats required by the new platform.
This includes identifying missing:
attributes
media
pricing rules
matching IDs
2. Build Wireframes That Show Data Requirements
Designing early page templates reveals exactly what data the site needs:
product detail
category landing
search
navigation
cart
checkout
3. Map Every Integration
For ERP, PIM, CRM, OMS, tax, shipping, or any third-party platform, determine:
what data flows in
what flows out
what the source of truth is
field-level requirements
frequency of sync
This prevents late-stage surprises during development.
Clean Data Makes Better Websites
A web project isn’t just about what your customers will see. It’s also about what your systems, workflows, and development team rely on to deliver a smooth experience.
Clean data enables:
faster builds
fewer bugs
more accurate search and navigation
better performance
smoother launch
lower long-term maintenance costs
Messy data does the opposite.
Whether you’re starting a project with us or researching how to prepare for one, investing time in data readiness is the single most impactful step you can take.
