Monday 9 February 2015

Using Zip files to publish hundreds of images at publish time

Sometimes you have a requirement to publish hundreds of images from Tridion as part of a page publish, my requirements were for a tool called Krpano http://krpano.com/ which generated a folder full of images, so the flow is as follows.

  1. generate all the images using the Krpano software.
  2. create a zip file of all the images
  3. create a Panorama component, and component link to the zip file.
  4. when the page is published all the images are extracted from the zip file and put in to the broker.

This is achieved with the following.


Multimedia schema "zip of images"



In our implementation we create the two metadata values in the schema, because we need them for the way we generate our code, but you can remove them, as they are added by the code if they are missing at publish time.

"Process Zip" Class

  • Create a new class that implements BaseTemplate, this looks for all component links etc that use the Multimedia Schema called "zip of images" and this uses the open source unzipping library http://dotnetzip.codeplex.com/
  • upload the assembly in to Tridion and create the accompanying TBB for Process Zip




Then you can drag the TBB in to your new component template in template builder


Make sure the debug level is set to info, there is plenty of debugging information output to aid you with diagnosing issues.




ProcessZip.cs

using System;
using System.Collections.Generic;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Text;
using System.Text.RegularExpressions;
using System.Xml;
using System.Xml.Serialization;
using DD4T.Templates.Base;
using DD4T.Templates.Base.Extensions;
using Ionic.Zip;
using Microsoft.Xml.Serialization.GeneratedAssembly;
using Tridion.ContentManager.CommunicationManagement;
using Tridion.ContentManager.ContentManagement;
using Tridion.ContentManager.Publishing;
using Tridion.ContentManager.Templating;
using Tridion.ContentManager.Templating.Assembly;
using Tridion.ContentManager.Workflow;
using Dynamic = DD4T.ContentModel;
using DD4T.ContentModel;
using Component = Tridion.ContentManager.ContentManagement.Component;
using Page = Tridion.ContentManager.CommunicationManagement.Page;
using TcmUri = Tridion.ContentManager.TcmUri;

namespace DD4T.Templates
{
    [TcmTemplateTitle("Process Zip")]

    class ProcessZip : BaseTemplate
    {


        private const string EmbargoedStatusName = "Embargoed";



        public override void Transform(Engine engine, Package package)
        {
            Log.Info(String.Format("Staring"));
            this.Log = Tridion.ContentManager.Templating.TemplatingLogger.GetLogger(this.GetType());



            this.Package = package;
            this.Engine = engine;

            var output = package.GetValue(Package.OutputName);

            if (String.IsNullOrEmpty(output))
            {
                throw new Exception("No output in package. This TBB will only work after some DD4T output has been created");
            }


            var stringBuilder = new StringBuilder();


            //Create our own namespaces for the output
            var ns = new XmlSerializerNamespaces();
            //Add an empty namespace and empty value
            ns.Add("", "");

            var xmlWriterSettings = new XmlWriterSettings
            {
                Encoding = new UTF8Encoding(),
                Indent = true,
                OmitXmlDeclaration = true
            };


            using (var writer = XmlWriter.Create(stringBuilder, xmlWriterSettings))
            using (var outputReader = new StringReader(output))
            {

                if (engine.PublishingContext.ResolvedItem.Item is Page)
                {
                    var serializer = new PageSerializer();
                    var page = (Dynamic.Page)serializer.Deserialize(outputReader);
                    this.FindZips(page.MetadataFields, this.CreateFilesFromZip);
                    serializer.Serialize(writer, page, ns);
                }
                else
                {
                    var serializer = new ComponentSerializer();
                    var component = (Dynamic.Component)serializer.Deserialize(outputReader);
                    this.FindZips(component, this.CreateFilesFromZip);
                    serializer.Serialize(writer, component, ns);
                }
            }

            var transformedOutput = stringBuilder.ToString();


            if (package.GetByName(Package.OutputName) != null)
            {
                var outputItem = package.GetByName(Package.OutputName);
                package.Remove(outputItem);
                package.PushItem(Package.OutputName, package.CreateStringItem(ContentType.Xml, transformedOutput));
            }
            else
            {
                package.PushItem(Package.OutputName, package.CreateStringItem(ContentType.Xml, transformedOutput));
            }










        }

        private void CreateFilesFromZip(Dynamic.Component component)
        {
            Component zipComponent = this.GetImageComponent(component.Id);
            Log.Info(String.Format("Zip component found {0}", zipComponent.Id));

            if (!zipComponent.BinaryContent.Filename.EndsWith(".zip", StringComparison.OrdinalIgnoreCase))
            {
                throw new Exception("Component is not a zip");
            }


            BinaryContent imageData = zipComponent.BinaryContent;



            using (MemoryStream ms = new MemoryStream(imageData.GetByteArray()))
            {
                Log.Debug("Opening Zip file in memory");

                using (ZipFile zip = ZipFile.Read(ms))
                {
                    var numberOfImagesInZipField = new Field
                    {
                        XPath = "",
                        FieldType = FieldType.Number,
                        Name = "numberOfImagesInZip",
                    };
                    var filePathsField = new Field
                    {
                        XPath = "",
                        FieldType = FieldType.Text,
                        Name = "filePaths",
                    };


                    int imageCount = 0;
                    List fileList = new List(zip.EntryFileNames);

                    //Use natural sort to allow them to add files with numbers in and sort in human readable order.
                    fileList.Sort(new NaturalSortComparer());

                    foreach (string file in fileList)
                    {
                        ZipEntry e = zip[file];
                        Log.Debug(String.Format("Extracted file from zip: {0}", e.FileName));
                        if (ValidFileToExtract(e.FileName))
                        {
                            using (MemoryStream extractedFileStream = new MemoryStream())
                            {
                                Component relatedComponent = (Component)Engine.GetObject(zipComponent.Id);
                                e.Extract(extractedFileStream);
                                string uniqueId = relatedComponent.Id.ToString().Replace(":", "");

                                string fileName = string.Format("{0}_{1}{2}", uniqueId, imageCount, Path.GetExtension(e.FileName));
                                Log.Debug(String.Format("Saving file as: {0}", fileName));


                                using (Bitmap extractedImage = new Bitmap(extractedFileStream))
                                {
                                    string addBinaryFilename = Path.GetFileName(fileName);
                                    string addBinaryVariantId = string.Format("{0}{1}", uniqueId, fileName);

                                    Log.Debug(String.Format("MimeType: {0},addBinaryFilename: {1},addBinaryVariantId: {2}", GetMimeType(extractedImage.RawFormat), addBinaryFilename, addBinaryVariantId));

                                    Engine.PublishingContext.RenderedItem.AddBinary(
                                                                            extractedFileStream, addBinaryFilename, addBinaryVariantId, relatedComponent,
                                                                            GetMimeType(extractedImage.RawFormat));
                                }
                                imageCount++;
                                filePathsField.Values.Add(fileName);

                            }


                        }
                    }


                    numberOfImagesInZipField.NumericValues.Add(imageCount);

                    //Remove the metadata values that are added in the CME so the webapp can generate the facade correctly
                    component.MetadataFields.Remove("numberOfImagesInZip");
                    component.MetadataFields.Remove("filePaths");

                    //Polpulate with the real values.
                    component.MetadataFields.Add("numberOfImagesInZip", numberOfImagesInZipField);
                    component.MetadataFields.Add("filePaths", filePathsField);


                }
            }
        }



        private void FindZips(Dynamic.Component component, Action callback)
        {
            if (component.Schema.Title == "zip of images")
            {
                callback(component);
            }
            else
            {
                this.FindZips(component.Fields, callback);
                this.FindZips(component.MetadataFields, callback);
            }
        }

        private void FindZips(FieldSet fieldSet, Action callback)
        {
            foreach (Field field in fieldSet.Values)
            {
                if (field.FieldType == FieldType.ComponentLink
                    || field.FieldType == FieldType.MultiMediaLink)
                {
                    foreach (var linkedComponent in field.LinkedComponentValues)
                    {
                        this.FindZips(linkedComponent, callback);
                    }
                }
                if (field.FieldType == FieldType.Embedded)
                {
                    foreach (var embeddedFields in field.EmbeddedValues)
                    {
                        this.FindZips(embeddedFields, callback);
                    }
                }
            }
        }


        private Component GetImageComponent(string tcmUri)
        {
            Log.Debug("Stepping into 'GetImageComponent'");

            Component imgComponent = null;

            //Solution to capture the in-worflow version
            string inWorkflowImageTcmUri = tcmUri + "-v0";

            //Is the page being previewed in Experience Manager?
            Boolean inExperienceManager = InExperienceManager();

            if (tcmUri.Contains("-v") || !Engine.GetSession().IsExistingObject(inWorkflowImageTcmUri))
            {
                Log.Debug("Looks like we are not in-workflow. Use last major version.");

                imgComponent = (Component)Engine.GetObject(tcmUri);
                if (IsEmbargoed(imgComponent))
                {
                    throw new Exception("Current component is embargoed");
                }
                if (!inExperienceManager && !MeetsMinimalApprovalStatus(imgComponent))
                {
                    throw new Exception("Current component does not meet minimal approval status");
                }

                return imgComponent;
            }

            if (inExperienceManager)
            {
                Log.Debug("Currently in Experience Manager");

                //This means we are in "Session Preview" (Experience Manager) mode
                return (Component)Engine.GetObject(inWorkflowImageTcmUri);
            }

            Log.Debug("Checking if in-workflow version can be used");

            imgComponent = (Component)Engine.GetObject(inWorkflowImageTcmUri);
            if (!IsEmbargoed(imgComponent) && MeetsMinimalApprovalStatus(imgComponent))
            {
                //Current in-workflow status meets the minimal approval requirement
                return imgComponent;
            }

            Log.Debug("Fallback and use the last major version");

            imgComponent = ((Component)Engine.GetObject(tcmUri)).GetPublishableVersion(Engine.PublishingContext.PublicationTarget);
            if (IsEmbargoed(imgComponent))
            {
                throw new Exception("Current component is embargoed");
            }
            if (!inExperienceManager && !MeetsMinimalApprovalStatus(imgComponent))
            {
                throw new Exception("Current component does not meet minimal approval status");
            }

            return imgComponent;
        }

        private Boolean MeetsMinimalApprovalStatus(Component imgComponent)
        {
            Log.Debug("Stepping into 'MeetsMinimalApprovalStatus'");

            //Get the image's approval status
            ApprovalStatus imgApprovalStatus = null;
            try
            {
                imgApprovalStatus = imgComponent.ApprovalStatus;
            }
            catch (Exception exception)
            {
                Log.Debug("Unable to fetch the image component's approval status [" + exception.Message + "]");
            }

            if (imgApprovalStatus == null)
            {
                //This means it's an image created before workflow was introduced
                Log.Debug("Returning 'true' since the image workflow status is 'undefined'");
                return true;
            }

            //Need to introduce this to support debugging with the Template Builder
            if (GetCurrentMode() == CurrentMode.TemplateBuilder)
            {
                Log.Debug("Returning 'true' since the Publication Target is 'null' in 'TemplateBuilder' mode");
                return true;
            }

            //Get the publication target's minimal approval status
            ApprovalStatus minApprovalStatus = Engine.PublishingContext.PublicationTarget.MinApprovalStatus;

            if (minApprovalStatus == null || imgApprovalStatus.Position >= minApprovalStatus.Position)
            {
                Log.Debug("Image status meets the minimal approval status!");
                return true;
            }

            Log.Debug("Image status does NOT meet the minimal approval status!");
            return false;
        }

        private Boolean InExperienceManager()
        {
            Log.Debug("Stepping into 'InExperienceManager'");

            CurrentMode currentMode = GetCurrentMode();
            if (currentMode == CurrentMode.SessionPreview)
            {
                //This means we are in Experience Manager
                return true;
            }

            return false;
        }

        private CurrentMode GetCurrentMode()
        {
            Log.Debug("Stepping into 'GetCurrentMode'");

            RenderMode renderMode = Engine.RenderMode;

            if (renderMode == RenderMode.Publish)
            {
                return CurrentMode.Publish;
            }

            if (renderMode == RenderMode.PreviewDynamic)
            {
                PublicationTarget pubTarget = Engine.PublishingContext.PublicationTarget;
                if (pubTarget == null)
                {
                    return CurrentMode.TemplateBuilder;
                }

                if (pubTarget.Id.Equals(TcmUri.UriNull))
                {
                    return CurrentMode.CmePreview;
                }

                return CurrentMode.SessionPreview;
            }

            return CurrentMode.Unknown;
        }

        private Boolean IsEmbargoed(Component imgComponent)
        {
            Log.Debug("Stepping into 'IsEmbargoed'");

            // Get the image's approval status
            ApprovalStatus imgApprovalStatus = null;
            try
            {
                imgApprovalStatus = imgComponent.ApprovalStatus;
            }
            catch (Exception exception)
            {
                Log.Debug("Unable to fetch the image component's approval status [" + exception.Message + "]");
            }

            if (imgApprovalStatus == null)
            {
                //This means it's an image created before workflow was introduced
                Log.Debug("Returning 'false' since the image workflow status is 'undefined'");
                return false;
            }

            Log.Debug("Image status is [" + imgApprovalStatus.Title + "]");
            if (String.Equals(imgApprovalStatus.Title, EmbargoedStatusName))
            {
                Log.Debug("Image is embargoed!");
                return true;
            }

            Log.Debug("Image is NOT embargoed!");
            return false;
        }


        private Boolean ValidFileToExtract(string fileName)
        {
            return fileName.ToLower().EndsWith(".jpg");
        }


        private static string GetMimeType(ImageFormat imageFormat)
        {
            //TODO: Could enumerate Tridion's configure mime types instead of the system's
            foreach (ImageCodecInfo imageCodecInfo in ImageCodecInfo.GetImageDecoders())
            {
                if (imageCodecInfo.FormatID == imageFormat.Guid)
                {
                    return imageCodecInfo.MimeType;
                }
            }
            return "image/unknown";
        }


        private enum CurrentMode
        {
            TemplateBuilder,
            CmePreview,
            SessionPreview,
            Publish,
            Unknown
        }

        public class NaturalSortComparer : IComparer, IDisposable
        {
            private bool isAscending;

            public NaturalSortComparer(bool inAscendingOrder = true)
            {
                this.isAscending = inAscendingOrder;
            }

            #region IComparer Members

            public int Compare(string x, string y)
            {
                throw new NotImplementedException();
            }

            #endregion

            #region IComparer Members

            int IComparer.Compare(string x, string y)
            {
                if (x == y)
                    return 0;

                string[] x1, y1;

                if (!table.TryGetValue(x, out x1))
                {
                    x1 = Regex.Split(x.Replace(" ", ""), "([0-9]+)");
                    table.Add(x, x1);
                }

                if (!table.TryGetValue(y, out y1))
                {
                    y1 = Regex.Split(y.Replace(" ", ""), "([0-9]+)");
                    table.Add(y, y1);
                }

                int returnVal;

                for (int i = 0; i < x1.Length && i < y1.Length; i++)
                {
                    if (x1[i] != y1[i])
                    {
                        returnVal = PartCompare(x1[i], y1[i]);
                        return isAscending ? returnVal : -returnVal;
                    }
                }

                if (y1.Length > x1.Length)
                {
                    returnVal = 1;
                }
                else if (x1.Length > y1.Length)
                {
                    returnVal = -1;
                }
                else
                {
                    returnVal = 0;
                }

                return isAscending ? returnVal : -returnVal;
            }

            private static int PartCompare(string left, string right)
            {
                int x, y;
                if (!int.TryParse(left, out x))
                    return left.CompareTo(right);

                if (!int.TryParse(right, out y))
                    return left.CompareTo(right);

                return x.CompareTo(y);
            }

            #endregion

            private Dictionary table = new Dictionary();

            public void Dispose()
            {
                table.Clear();
                table = null;
            }
        }
    }
}

Output after running past the ProcessZip TBB

As you can see the two metadata fields for the zip are now populated numberOfImagesInZip contains a count of images found and extracted from each zip, and filePaths contains a list of all the filepaths releative to the root images folder for each file out of the zip extracted.
                      
                        
                          numberOfImagesInZip
                          
                            numberOfImagesInZip
                            
                            
                              10
                            
                            
                            
                            
                            
                          
                        
                        
                          filePaths
                          
                            filePaths
                            
                              tcm1273-61578_0.jpg
                              tcm1273-61578_1.jpg
                              tcm1273-61578_2.jpg
                              tcm1273-61578_3.jpg
                              tcm1273-61578_4.jpg
                              tcm1273-61578_5.jpg
                              tcm1273-61578_6.jpg
                              tcm1273-61578_7.jpg
                              tcm1273-61578_8.jpg
                              tcm1273-61578_9.jpg
                            
                            
                            
                            
                            
                            
                          
                        
                      


No comments:

Post a Comment