Deep-dive into cloud-native AEM deployments based on Kubernetes

Please download to get full document.

View again

of 36
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Similar Documents
Information Report
Category:

Software

Published:

Views: 0 | Pages: 36

Extension: PDF | Download: 0

Share
Description
AEM instance is traditionally see as a "pet" - an application that requires manual work for the deployment and the constant oversee in the runtime. Transforming instances into "cattle" and moving them into cloud brings a number of challenges around persistence, replication, scaling, monitoring, upgrades and maintenance. However, cloud computing services like Azure and container orchestration systems like Kubernetes allows to solve these problems in the new, creative ways, while the resulting cloud setup offers scalability not possible before. This session will provide an overview on the Kubernetes setup we internally use in Adobe, the issues we've run into and the ways we're dealing with them.
Transcript
  • 1. APACHE SLING&FRIENDSTECHMEETUP 2- 4SEPTEMBER 2019 Deep-diveinto cloud-native AEM deployments based on Kubernetes Tomek Rękawek, Adobe
  • 2. 2 Cloud: whatand why?
  • 3. Wewanttomovefrom 3
  • 4. to 4
  • 5. Cloud:why? 5  RunningAEM at scale  Hassle-freedeployments  The cloud provider (AWS, Azure) should worry about infrastructure
  • 6. Agenda 6  Jobs  Content migration  New indexes  QA  Docker&Kubernetesintroduction  Architectureoverview  Publishpersistence  Cloud Segment Store  Golden publish  Compaction  Sidecarservices
  • 7. 7 Docker & Kubernetesintroduction
  • 8. Thepowerofstandardization 8
  • 9. DockerizedAEM 9  AEM in Docker image  Composite Node Store  /apps& /libs storedin thecontainer  Actualcontentlives outside,in the VOLUMEorMongoDB  OSGi Feature Model  DefinesAEMandcustomerapplication  Featurelauncherstartsit inside container  Covered in Karl& David’spresentation  SeeadaptTo() 2017 talk for moredetails
  • 10. Mini intro toKubernetes 10  Launches Docker containers without worrying about the underlying VMs  Dictionary  Pod–a groupofcontainersstartingtogetheron asingle machine  Service –internalloadbalancer,exposinganumberofpodreplicas underasingle address  Ingress– exposesa service underanpublic httpaddress  Everyentity is represented by a YAML object in K8s API server  TheYAML is the desired state, K8s knows how to get there
  • 11. DeploymentswithHelm 11  Helm – K8s apps management  Chart – a bunch of K8s YAML descriptors, with a simple templating  Whole AEMdeployment canbe installed/upgraded with a single command Image:https://helm.sh
  • 12. 12 Architecture overview
  • 13. AEMKubernetessetup 13
  • 14. 14 Publishpersistence evolution
  • 15. Problemdefinition 15  Author persistence is easy-ish with MongoDB  Publish is harder –local SegmentMK, no clustering  Thepublish farm is kept up-to-date with replication  However:  weneed toprovidethenew publishinstanceswitha segment store,  copyit fromanotherinstance.  Problems:  copyingfiles betweenpodsis hardandhacky,  whatif there’snopublishto copyfrom?
  • 16. Cloud SegmentStore 16  A new plugin for the Segment Node Store  Nodes are stored in a cloud storage service  No tar files, raw segments grouped in dirs  Can be used in RW or RO modes
  • 17. Goldenpublish 17  A designated publish instance  Not connectedto LB  It maintainsa “golden copy” of the segmentstore  New publish just clone it
  • 18. Problem:duplicatedbinaries andstartuptime 18 Golden publish Segment 1 Segment 2 Segment 4 Segment 3 Segment x Publish 1 Segment 1 Segment 2 Segment 4 Segment 3 Segment y Publish 2 Segment 1 Segment 2 Segment 4 Segment 3 Segment z Publish 3 (starting) Segment 1 Segment 2 Segment 3 Segment 4  Multiple copies of the same segments 1-4 ($$$)  Cloning a bucket takes time during the publish start ( 🕑🕑🕑)
  • 19. Optimization:a singlesegmentstore 19
  • 20. 20 Publishpersistence: compaction
  • 21. Compaction 21
  • 22. Out-of-bandpublish update 22  This pattern willbe usefulin many cases  We may clone thepublish repository, modify it and re-deploy instanceson top of it  Persisted message queuewill apply missingchanges
  • 23. 23 Sidecar services
  • 24. Sidecar approach 24  A single pod can run many containers, sharing their volumes and localhost interface  We can use them for the auxiliaryservices (sidecars):  Dispatcher in the publish pod  Upload logs to Splunk  Warmup service
  • 25. AEMpod withsidecars 25Sidecaricon: Flaticons.com
  • 26. 26 Jobs
  • 27. Kubernetesjob 27  Starts a pod  Meantto perform a specific task and finish  Unlikethe deployment, which run indefinitely  Willbe restarted if fails  Jobs provides a way to interactwiththe deployed AEM
  • 28. 28 Contentmigration
  • 29. Contentmigration 29  How to migrate old content to K8s?  Access problem  old AEM envs shouldn’t have access to the K8s (encapsulation)  K8s shouldn’t be able to access oldAEMs (they can be installed anywhere)  Solution:demilitarized zone
  • 30. 2-phasemigration 30  Phase 1  Themigrator tool (crx2oak-like jar) is used to export old AEM content into a cloud storage service  Phase 2  A Kubernetes job is used to apply the migrated content on the cloud instances
  • 31. 31 Newindexes
  • 32. Addingnew index 32  Indexesaretricky  /oak:index is a part of the mutable content  But indexdefinitions belongs to the application  Onlyaddingnew indexesis supported  Whena new indexis addedin /oak:index,it’ll haveanextra useIfExistspropertyreferencing immutable part/apps  This bounds the index definition tothe application version andDocker image  This /appspathhaveto beaddedas well • /oak:index/my-new-index • useIfExists: /apps/indexes@v3 • /apps/indexes • v3: true Mutable part Immutable part
  • 33. Mutablecontent:addingnew index 33  Indexingjob should be runbefore the actual app deployment  Thejob will: 1. Look forthe new index defs in /oak:index 2. Perform out of band indexing ofthe content 3. Save the new indexing content tothe production repository  TheuseIfExists will makesure that the newindexis ignored, untilthe newimageis installed  When the new image is deployed, the /apps part will be updated and the new index will beused
  • 34. 34 Other topics
  • 35. Relatedtopics 35  Feature model usage in Docker (covered in David’s and Karl’stalk)  Replication (covered in Timothee’s talk)  Monitoring with Prometheus and Grafana  CI/CD pipeline  Network policies  …
  • 36. 36 Thanks!
  • We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks
    SAVE OUR EARTH

    We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

    More details...

    Sign Now!

    We are very appreciated for your Prompt Action!

    x