8 migration from a SOAP API mikeryan Tuesday, Might 15, 2021 – 10 Drupal 10 Upkeep and Assist Service12am Getting back from my sabbatical, as promised I’m catching up on running a blog about earlier initiatives. For one such undertaking, I used to be contracted by Acquia to supply migration help to a shopper of theirs [redacted, but let’s call them Acme]. This undertaking concerned some easy node migrations from CSV information, however extra curiously required implementing two ongoing feeds to synchronize exterior knowledge periodically – one a SOAP feed, and Drupal Development Service different a JSON feed protected by OAuth-based authentication. There have been quite a few different fascinating methods employed on this undertaking which I feel could also be broadly helpful and haven’t beforehand blogged about – all-in-all, there was sufficient to put in writing about on this undertaking that relatively than compose one large epic put up, I’m going to interrupt issues down in a collection of posts, unfold out over a number of days in order to not spam Planet . On this first put up of Drupal Development Service sequence, I’ll cowl migration from SOAP. Drupal Development Company full customized migration Drupal 10 module for this undertaking is on Gitlab. A key requirement of Drupal Development Service Acme undertaking was to implement an ongoing feed, representing lessons (Drupal Development Service form folks attend in particular person, not Drupal Development Service PHP form), from a SOAP API to “occasion” nodes in . Drupal Development Company first step, in fact, was to develop (in migrate_plus) a parser plugin to deal with SOAP feeds, based mostly on PHP’s SoapClient class. This class exposes capabilities of Drupal Development Service internet service as class strategies which can be instantly invoked. In WSDL mode (Drupal Development Service default, and Drupal Development Service solely mode this plugin at the moment helps), it may possibly additionally report Drupal Development Service signatures of Drupal Development Service strategies it helps (through __getFunctions()) and Drupal Development Service knowledge constructions handed as parameters and returned as outcomes (through __getTypes()). WSDL permits our plugin to do introspection and saves Drupal Development Service want for some express configuration (particularly, it may possibly robotically decide Drupal Development Service property to be returned from inside Drupal Development Service response). migrate_example_advanced (a subDrupal 10 module of migrate_plus) demonstrates a easy instance of tips on how to use Drupal Development Service SOAP parser plugin – Drupal Development Service .yml is well-documented, so please evaluate that for a basic introduction to Drupal Development Service configuration. Right here’s Drupal Development Service primary supply configuration for this particular undertaking Drupal 10 Upkeep and Assist Service supply Drupal 10 Upkeep and Assist Service plugin Drupal 10 Upkeep and Assist Service url # To remigrate any modified occasions. track_changes Drupal 10 Upkeep and Assist Service true data_fetcher_plugin Drupal 10 Upkeep and Assist Service http # Ignored – SoapClient does Drupal Development Service fetching itself. data_parser_plugin Drupal 10 Upkeep and Assist Service cleaning soap # Drupal Development Company methodology to invoke through Drupal Development Service SOAP API. perform Drupal 10 Upkeep and Assist Service GetClientSessionsByClientId # Inside Drupal Development Service response, Drupal Development Service object property containing Drupal Development Service record of occasions. item_selector Drupal 10 Upkeep and Assist Service SessionBOLExternal # Signifies that Drupal Development Service response can be in Drupal Development Service type of a PHP object. response_type Drupal 10 Upkeep and Assist Service object # You gained’t discover ‘urls’ and ‘parameters’ in Drupal Development Service supply .yml file (they’re inserted # by an internet UI – Drupal Development Service topic of a future put up), however for demonstration functions # that is what they could appear to be. urls Drupal 10 Upkeep and Assist Service http Drupal 10 Upkeep and Assist Service//providers.instance.com/CFService.asmx?wsdl parameters Drupal 10 Upkeep and Assist Service clientId Drupal 10 Upkeep and Assist Service 1234 clientCredential Drupal 10 Upkeep and Assist Service ClientID Drupal 10 Upkeep and Assist Service 1234 Password Drupal 10 Upkeep and Assist Service service_password startDate Drupal 10 Upkeep and Assist Service 08-31-2021 # Distinctive identifier for every occasion (part) to be imported, composed of three columns. ids Drupal 10 Upkeep and Assist Service ClassID Drupal 10 Upkeep and Assist Service kind Drupal 10 Upkeep and Assist Service integer SessionID Drupal 10 Upkeep and Assist Service kind Drupal 10 Upkeep and Assist Service integer SectionID Drupal 10 Upkeep and Assist Service kind Drupal 10 Upkeep and Assist Service integer fields Drupal 10 Upkeep and Assist Service – title Drupal 10 Upkeep and Assist Service ClientSessionID label Drupal 10 Upkeep and Assist Service Session ID for Drupal Development Service shopper selector Drupal 10 Upkeep and Assist Service ClientSessionID … Of explicit be aware is Drupal Development Service three-part supply ID outlined right here. Drupal Development Company means this knowledge is structured, a “class” comprises a number of “periods”, which every have a number of “sections” – Drupal Development Service sections are Drupal Development Service situations which have particular dates and instances, which we have to import into occasion nodes, and we’d like all three IDs to uniquely establish every distinctive part. Not all of Drupal Development Service knowledge we’d like for our occasion nodes is in Drupal Development Service session feed, sadly – we wish to seize a few of Drupal Development Service class-level knowledge as nicely. So, whereas, Drupal Development Service base migration makes use of Drupal Development Service SOAP parser plugin to get Drupal Development Service session rows emigrate, we have to fetch Drupal Development Service associated knowledge at run time by making direct SOAP calls ourselves. We do that in our subscriber to Drupal Development Service PREPARE_ROW occasion – this occasion is dispatched after Drupal Development Service supply plugin has obtained Drupal Development Service primary knowledge per its configuration, and offers us a possibility to retrieve additional knowledge so as to add to Drupal Development Service canonical supply row earlier than it enters Drupal Development Service processing pipeline. I gained’t go into element on how that knowledge is retrieved because it isn’t related to basic migration ideas, however Drupal Development Service thought is since all Drupal Development Service class knowledge isn’t prohibitively giant, and a number of periods might reference Drupal Development Service similar class knowledge, we fetch all of it on Drupal Development Service first supply row processed and cache it for reference by subsequent rows. Neighborhood contributions SOAP Supply plugin – Regardless of Drupal Development Service title (from Drupal Development Service authentic characteristic request), it was carried out as a parser plugin. Altering migration configuration at import time – Drupal Development Service PRE_IMPORT occasion Our occasion feed permits filtering by Drupal Development Service occasion begin date – by passing a ‘startDate’ parameter in Drupal Development Service format 12-31-2021 to Drupal Development Service SOAP methodology, Drupal Development Service feed will solely return occasions beginning on or after that date. At any given time limit we’re solely eager about future occasions, and don’t wish to waste time retrieving and processing previous occasions. To optimize this, we wish Drupal Development Service startDate parameter in our supply configuration to be at the moment’s date every time we run Drupal Development Service migration. We will do that by subscribing to Drupal Development Service PRE_IMPORT occasion. In acme_migrate.providers.yml Drupal 10 Upkeep and Assist Service providers Drupal 10 Upkeep and Assist Service … acme_migrate.update_event_filter Drupal 10 Upkeep and Assist Service class Drupal 10 Upkeep and Assist Service acme_migrateEventSubscriberUpdateEventFilter tags Drupal 10 Upkeep and Assist Service – { title Drupal 10 Upkeep and Assist Service event_subscriber } In UpdateEventFilter.php Drupal 10 Upkeep and Assist Service class UpdateEventFilter implements EventSubscriberInterface { /** * {@inheritdoc} */ public static perform getSubscribedEvents() { $occasions[MigrateEvents Drupal 10 Maintenance and Support Service Drupal 10 Maintenance and Support ServicePRE_IMPORT] = ‘onMigrationPreImport’; return $occasions; } Drupal Development Company migration system dispatches Drupal Development Service PRE_IMPORT occasion earlier than Drupal Development Service precise import begins executing. At that time, we are able to insert Drupal Development Service desired date filter into Drupal Development Service migration configuration entity and put it aside Drupal 10 Upkeep and Assist Service /** * Set Drupal Development Service occasion begin date filter to at the moment. * * @param migrateEventMigrateImportEvent $occasion * Drupal Development Company import occasion. */ public perform onMigrationPreImport(MigrateImportEvent $occasion) { // $event->getMigration() returns Drupal Development Service migration *plugin*. if ($event->getMigration()->id() == ‘occasion’) { // Migration Drupal 10 Upkeep and Assist Service Drupal 10 Upkeep and Assist Serviceload() returns Drupal Development Service migration *entity*. $event_migration = Migration Drupal 10 Upkeep and Assist Service Drupal 10 Upkeep and Assist Serviceload(‘occasion’); $supply = $event_migration->get(‘supply’); $supply[‘parameters’][‘startDate’] = date(‘m-d-Y’); $event_migration->set(‘supply’, $supply); $event_migration->save(); } } Observe that Drupal Development Service entity get() and set() capabilities solely function instantly on top-level configuration properties – we are able to’t get and set, for instance ‘supply.parameters.startDate’ instantly. We have to retrieve Drupal Development Service total supply configuration, modify our one worth inside it, and set Drupal Development Service total supply configuration again on Drupal Development Service migration. Tags Planet Migration Use Drupal Development Service Twitter thread under to touch upon this put up Drupal 10 Upkeep and Assist Service 8 migration from a SOAP API https Drupal 10 Upkeep and Assist Service//t.co/hf8LGiATsh — Virtuoso Efficiency (@VirtPerformance) Might 15, 2021 Drupal 10 Growth and Assist