Normally, migrations get run at Drupal Developer starting of a undertaking to get all of your content material into . However typically, it’s essential create a migration that runs regularly. For instance, you’ve got an exterior database of programs and applications maintained by a college that must be displayed on a web site. One other instance Drupal 10 Upkeep and Assist Service you’ve got a database of ebook knowledge that must be pulled into nightly. When working with migrations the place Drupal Developer supply information are up to date each day, it might get actually tedious to obtain Drupal Developer up to date supply information manually every time Drupal Developer migration runs. On this tutorial, we’ll write a supply plugin primarily based on Drupal Developer CSV supply plugin which is able to enable us to routinely obtain CSV information from a distant server by way of SFTP earlier than working migrations. This text was co-authored by my colleague David Valdez – gracias David in your contribution. Drupal Development Downside In a undertaking we labored on not too long ago, we had Drupal Developer following scenario Drupal 10 Upkeep and Assist Service CSV information are up to date by a PowerShell script each evening on Drupal Developer shopper’s server. These CSV information are accessible by way of SFTP. Our process is to obtain Drupal Developer CSV supply information over SFTP and to make use of them as our migration supply. Earlier than We Begin This articles assumes you can write customized Drupal 10 modules. When you have by no means written a customized Drupal 10 module, you possibly can attempt studying this text on creating customized Drupal 10 modules. It’s assumed that you’ve working information of migrations in 8. In case you are new to 8 migrations, I like to recommend you to start out by studying these articles first Drupal 10 Upkeep and Assist Service 8 Migration Drupal 10 Upkeep and Assist Service Migrating Fundamental Knowledge (Half 1) 8 Migration Drupal 10 Upkeep and Assist Service Migrating Taxonomy Time period References (Half 2) 8 Migration Drupal 10 Upkeep and Assist Service Migrating Recordsdata / Pictures (Half 3) Drupal Development Plan Drupal Development aim is to keep away from downloading Drupal Developer file manually each time we run our migrations. So we’d like a technique to doing this routinely everytime we execute a migration. To realize this, we create a customized supply plugin extending Drupal Developer CSV plugin offered by Drupal Developer Migrate Supply CSV Drupal 10 module, which is able to obtain CSV information from a distant server and go it to Drupal Developer CSV plugin to course of them. Drupal Development Supply Migrate Plugin To start out, let’s create a customized Drupal 10 module and name it migrate_example_source and implement a customized migrate supply plugin by making a PHP class inside it at /src/Plugin/migrate/supply/MigrateExampleSourceRemoteCSV.php We begin implementing Drupal Developer class by merely extending Drupal Developer CSV plugin offered by Drupal Developer migrate_source_csv Drupal 10 module Drupal 10 Upkeep and Assist Service namespace migrate_source_csvPluginmigratesource; use migrate_source_csvPluginmigratesourceCSV as SourceCSV; use phpseclibNetSFTP /** * @MigrateSource( * id = “migrate_example_source_remote_csv” * ) */ class MigrateExampleSourceRemoteCSV extends SourceCSV {}Including Drupal Developer annotation @MigrateSource is essential as a result of that’s what will make Drupal Developer migrate Drupal 10 module detect our supply plugin. In our plugin, we use Drupal Developer phpseclib/phpseclib libraries to make SFTP connections. Therefore, we have to embody Drupal Developer libraries in our undertaking by working Drupal Developer following command in Drupal Developer root Drupal 10 Upkeep and Assist Service composer require phpseclib/phpseclibOur plugin will obtain Drupal Developer supply CSV file and can merely go it to Drupal Developer CSV plugin to do Drupal Developer relaxation. We do Drupal Developer obtain when Drupal Developer plugin is being instantiated like this Drupal 10 Upkeep and Assist Service /** * {@inheritdoc} */ public operate __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration) { // If SFTP connection parameters are current. if (!empty($configuration[‘sftp’])) { // A settings key have to be specified. // We use Drupal Developer settings key to get SFTP configuration from $settings. if (!isset($configuration[‘sftp’][‘settings’])) { throw new MigrateException(‘Parameter “sftp/settings” not outlined for Distant CSV supply plugin.’); } // Merge plugin settings with international settings. $configuration[‘sftp’] += Settings Drupal 10 Upkeep and Assist Service Drupal 10 Upkeep and Assist Serviceget(‘sftp’, []); // We merely obtain Drupal Developer distant CSV file to a short lived path and set // Drupal Developer non permanent path to Drupal Developer father or mother CSV plugin. $configuration[‘path’] = $this->downloadFile($configuration[‘sftp’]); } // Having downloaded Drupal Developer distant CSV, we merely go Drupal Developer name to Drupal Developer father or mother plugin. father or mother Drupal 10 Upkeep and Assist Service Drupal 10 Upkeep and Assist Service__construct($configuration, $plugin_id, $plugin_definition, $migration); }In Drupal Developer constructor we’re utilizing international SFTP credentials with Settings Drupal 10 Upkeep and Assist Service Drupal 10 Upkeep and Assist Serviceget(). We have to outline Drupal Developer credentials in settings.php like this Drupal 10 Upkeep and Assist Service $settings[‘sftp’] = array( ‘default’ => [ ‘server’ => ‘ftp.example.com’, ‘username’ => ‘username’, ‘password’ => ‘password’, ‘port’ => ’22’, ], );As soon as we’ve got Drupal Developer credentials of Drupal Developer FTP server we use a downloadFile() technique to obtain Drupal Developer distant CSV file. This is an extract of Drupal Developer related code Drupal 10 Upkeep and Assist Service protected operate downloadFile(array $conn_config) { … // Put together to obtain file to a short lived listing. $path_remote = $conn_config[‘path’]; $basename = basename($path_remote); $path_local = file_directory_temp() . ‘/’ . $basename; … // Obtain file by SFTP and place it in non permanent listing. $sftp = static Drupal 10 Upkeep and Assist Service Drupal 10 Upkeep and Assist ServicegetSFTPConnection($conn_config); if (!$sftp->get($path_remote, $path_local)) { throw new MigrateException(‘Can’t obtain distant file ‘ . $basename . ‘ by SFTP.’); } … // Return path to Drupal Developer native of Drupal Developer file. // This can in flip be handed to Drupal Developer father or mother CSV plugin. return $path_local; }Word Drupal 10 Upkeep and Assist Service Drupal Development code block above has been simplified a bit. For those who see Drupal Developer precise supply plugin, there are some strains of code which make issues extra appropriate with Drupal Developer migration_lookup plugin. This technique creates an SFTP connection, downloads Drupal Developer file to a short lived location and returns Drupal Developer path to Drupal Developer downloaded file. Drupal Development non permanent file path is then handed to Drupal Developer Migrate Supply CSV and that is it! Lastly, to make use of Drupal Developer plugin in our migration we simply set our plugin as Drupal Developer supply/plugin Drupal 10 Upkeep and Assist Service id Drupal 10 Upkeep and Assist Service migrate_example_content label Drupal 10 Upkeep and Assist Service ‘Instance content material’ … supply Drupal 10 Upkeep and Assist Service plugin Drupal 10 Upkeep and Assist Service migrate_example_source_remote_csv # Settings for our customized Distant CSV plugin. sftp Drupal 10 Upkeep and Assist Service settings Drupal 10 Upkeep and Assist Service sftp path Drupal 10 Upkeep and Assist Service “/path/to/file/example_content.csv” # Settings for Drupal Developer contrib CSV plugin. header_row_count Drupal 10 Upkeep and Assist Service 1 keys Drupal 10 Upkeep and Assist Service – id …Drupal Development code for this plugin and Drupal Developer instance Drupal 10 module is on the market at migrate_example_source. Nice! + extra superior articles by Evolving Net Drupal 10 Growth and Assist