I am working on notes for a draft that will be a book about migration processes made with Drupal and its Migrate API. It is expected to be released in June 2020 and the work of collecting, experimenting and articulating the content is being quite extensive. As there are still some months left for the launch, in order not to lose the mental sanity and to be able to give partial sense to these tasks, I have thought to publish here some small posts derived from the working notes.
This way I will be able to give something useful to the complementary notes and if the COVID-19 attacks me before seeing the book come out, at least I will have shared something before (I guess).
Well, what do I want to talk about in this post? I would like to make a list of Drupal modules related to migration processes, available as contrib modules and that can be used to provide functionality to a migration. This article will be only a lightweight set of basic resources (I swear).
Table of Contents
2- Basic Resources - Core Modules
3- Other Basic Resources - Contrib Modules
4- Extra Resources - Contrib Modules for Plugins
5- Migration Runners - Contrib Modules Drush-Related
6- Authors you should know
This article is part of a series of posts about Drupal Migrations:
It’s not very easy to talk about migrations in general and, of course, it is not easy in the context of Drupal either. To perform migrations it is necessary to have a good knowledge of the technology, data models (in origin and in destination), experience in ETL processes and a certain know-how about how to implement Drupal Plugins (In migrations there is an extensive use of Drupal-Way Plugins).
In any case, since the topic is extensive and my time is now short, I thought of this article as a summary catalogue (for quick consumption) of tools and basic resources for working with migrations.
Migrate: The Mainframe for migrations, the migrate, that provides the base API for migrating in Drupal.
Migrate Drupal: migrate_drupal. Focused on upgrades from Drupal 6 or Drupal 7. Allow reading of configuration entities in Drupal 8.
Migrate Drupal UI: migrate_drupal_ui. Interface for upgrading.
Migrate Plus: [migrate_plus](https://www.drupal .org/project/migrate_plus). Migrate Plus is an essential contrib module wich extends the features and capabilities of the Migrate core module with a lot of plugins and extensions.
Migrate Tools: migrate_tools. Another essential resource: provides a lot of Drush commands for running and managing Migrations.
Migrate Status: migrate_status. This little contrib module allows get a feedback about a migration process. Do you need to know if a migration is running? this module gives you a service that you can call in order to check the migration.
Migrate Files: migrate_files. It’s such an interesting set of process plugins that you will want to move files and images with it.
Migrate Commerce: commerce_migrate. General-purpose framework that extends to the main Migrate module from Drupal Core, for moving data in a Drupal Commerce scenario.
In the Drupal migration processes, we’ll use diverse resources in order to processing the ETL migration plan. One of these basic resources (as I mentioned in the introduction) are the Drupal Plugins, of which you need to have good knowledge and some practice. In a migration scenario, plugins help us processing information from the E:Source (Source Plugins), making T:Processing(ProcessPlugins) in order to save data at L:Destination (Destination Plugins). The assembly of these three parts (usually) results in a correct migration process.
Many modules of the core already bring their own Plugins to facilitate migration processes (as the user module). So let’s review some migration plugins packaged in contributed modules.
Migrate Source CSV: migrate_source_csv. Contrib Module for migrating data to Drupal 8 from a classical and simple CSV file.
Migrate Source SQL: custom_sql_migrate_source_plugin. As a peculiarity of the Plugins used for databases, this module allows to integrate in the .yml file describing a migration, directly SQL queries that will be executed against the source database.
Migrate Source YAML: migrate_source_yaml. It’s just a simple tool for migrating content from YAML files.
Migrate Process XML: migrate_process_xml. Provides process plugins for xpath and xvalue.
Migrate HTML to Paragraphs: migrate_html_to_paragraphs. Helps to transform HTML from a migration Source in a Paragraph item (managed by a Destination Plugin).
What kind of Drupal entities will be created in the migrating process? content entities? configuration entities? Take a look.
Migrate Destination CSV: d8migrate. It’s a light custom module created by @jonathanfranks.
Migrate Destination Config: Class Config.php. Offers a plugin for config migration.
Migrate Destination Block: Class EntityBlock.php. Just like and example about the resources that every element can offers in a migration scene, in case of moving Block Entities (are Config Entities) see the PHP classes included in its own module for migrating (Source, Process and Destination).
Migrate Scheduler: migrate_scheduler. This module offers integration with the Drupal Cron API to execute migrations under predefined schedules.
Migrate Run: migrate_run. Drush commands to running migrations in a lightweight mode. More lean than Migrate Tools but not offer support for migrations groups. It also doesn’t depend on the Migrate Plus module. It’s just like a little maverick.
Migrate Devel: migrate_devel. Provides Drush options in order to show debug info while executing migrations. Also provides the ‘debug’ process plugin. Here you can see an article about it. By the way, the Drush 9 compatibility is still unresolved, but a patch seems to be available.
An essential training material. In addition, his company’s website, under the tag “migrate” also hosts many very good articles about migration topics: https://agaric.coop/tags/migrate.
Some examples from Mauricio Dinarte:
His profile in Drupal.org: https://www.drupal.org/u/dinarcon.
Tess Flynn: I heard about Tess Flynn reading articles by Mauricio Dinarte. That’s how I met this expert developer, speaker and communicator of the Drupal community. On her website https://deninet.com I found content of a different nature, but above all, a series of very interesting articles about migrations under the tag “drupal-migration”: https://deninet.com/tag/drupal-migration.
Along the way I also discovered that it has several contrib modules related to Migrations and Processing Plugins.
Some examples from Tess Flynn:
Her profile in Drupal.org: https://www.drupal.org/u/socketwench.
Some examples from Danny Sipos:
His profile in Drupal.org: https://www.drupal.org/u/upchuk.
De la pradera tiene musho peligro al ataquerl te voy a borrar el cerito torpedo tiene musho peligro pupita al ataquerl diodeno. Torpedo ese pedazo de qué dise usteer a peich ese que llega la caidita pecador.