This documentation is specific for Camelot SharePoint Integration Toolkit version 2.0.2. Documentation for earlier versions can be found here.


Prerequisites

This project requires the Camelot .NET Connector and at least a development license, which are free and renewable for all. To download the connector, visit http://www.bendsoft.com/net-sharepoint-connector/download/ and follow the instructions. You will need to register (with no obligations whatsoever) to get your free development license.


Installation 

Camelot Integration Toolkit Setup     Camelot Integration Toolkit Setup

  • When, complete, the setup will ask you if you wish to start the "Camelot SharePoint Integration Service" and test the service. For your convenience, check both and finish installation. The service will then be automatically started and tested through your browser. If everything is fine, the CamelotService page will show up. If something went wrong, you probably forgot to install the Camelot .NET Connector.

Camelot Integration Toolkit Setup


Reconfiguring the WCF service

Before testing the WCF service, you will need to configure the endpoint of the service and setup connectionstrings for the connector.

  • From the Windows start menu, locate the "Camelot SharePoint Integration Toolkit" under programs. Stop the WCF service by clicking "STOP Camelot SharePoint Integration Service".
  • From the location, open the application configuration associated to the integration service, click "Application XML Configuration". You will need a suitable XML editor installed to open this file.

First, modify the shared key. This key is shared between the service and all clients and must be provided when calling the service.

  • Under appSettings, modify the value of "SHARED_KEY" to the key of your choice

Modify CDATA option (optional). This setting indicates whether text in CamelotXml's are enclosed in CDATA brackets or with traditional xml escaping (default)

  • Under appSettings, modify the value of "WRITE_CDATA" to true if you want to enable CDATA brackets.

Modify ENABLE_IMPERSONATION option (optional). This indicates if the Camelot WCF service is allowed to impersonate the calling user. If NTML/Windows authentication is configured for the WCF service, the caller will need to authenticate to the WCF service. This service will then impersonate this user in all calls to SharePoint (requires that the connection string is setup with default authentication), ensuring that the user can only access lists and document libraries where permission is granted. See the next section for details on how to configure the WCF service authentication.

  • Under appSettings, modify the value of "ENABLE_IMPERSONATION" to true (default) if you want to enable impersonation. If this value is true, but no authentication scheme has been configured, the WCF service will not try to impersonate the user.

 

 

Setup connection strings! Now, this part may need som explanation for new users. The Camelot WCF service uses the Camelot .NET Connector for querying SharePoint lists, views and document libraries. The connector is a ADO.NET driver for Microsoft SharePoint that allows querying SharePoint using standard SQL syntax, kind of using SharePoint as a typical database. Similar do other databases, this connector requires a connection string for connecting to SharePoint (see connection string options).

In the Camelot WCF service, connection strings are stored in the application configuration file. When calling the Camelot WCF service, the caller must specify the name of the connection string to use. You can configure as many different connection strings as you wish in the configuration.

Camelot Integration Toolkit Setup

  • Under connectionstrings, copy or modify the existing connection string called "sharepoint_connection". In the simpliest case, you should only need to change Server, Database, User and Password. Server is the ip or hostname to your SharePoint server and username/password must be valid SharePoint credentials. The database option is the site path on the SharePoint server, for example "HR" or "Development/Documentation".

NOTE! If your SharePoint server uses SSL (HTTPS), you must enable SSL in the connector by adding SSL=True the connection string.

Configure the endpoint of the WCF service. In the configuration, locate the services section. You may need to modify the default base address (http://localhost:8080) according to your needs. This will be the address to the Camelot WCF service.

  • If necessary, modify value baseAddress.

Save the configuration file and restart the WCF service.

  • From the Windows start menu, click "START Camelot SharePoint Integration Service"

Using NTLM/Windows authentication in the WCF service

To enable NTLM/Windows authentication in the Camelot WCF Service, you will first need to modify the security mode in the binding options.

  • From the Windows start menu, locate the "Camelot SharePoint Integration Toolkit" under programs. Stop the WCF service by clicking "STOP Camelot SharePoint Integration Service".
  • From the location, open the application configuration associated to the integration service, click "Application XML Configuration". You will need a suitable XML editor installed to open this file.

Under basicHttpBinding, replace security mode "None" to "TransportCredentialsOnly" as shown below. Under appSettings, modify the value of "ENABLE_IMPERSONATION" to true (default) to enable impersonation.

<!--
<security mode="None"/>
-->

<security mode="TransportCredentialOnly">
     <transport clientCredentialType="Ntlm"/>
</security>

Now the service will require username and password authentication. This means that the service will impersonate the caller when querying SharePoint, ensuring that the caller can only read from and write to lists and document libraries according to the user's permissions in SharePoint. However, for this to work, the connection strings in the WCF service must be changed to authenticate with the default scheme. Explicit user credentials in connection strings will bypass this functionality.

Under connectionStrings, change the connection string by removing User and Password and replace Authentication with Default. This instructs the Camelot SharePoint Connector to connect with SharePoint as running user.

<connectionStrings>
  <!--<add name="sharepoint_connection" connectionString="Server=yoursharepointserver;Database=cms;User=spuser;Password=sppassword;Authentication=Ntlm;TimeOut=60;" />-->
  <add name="sharepoint_connection" connectionString="Server=yoursharepointserver;Database=cms;Authentication=Default;TimeOut=60;" />
</connectionStrings>

To finish the configuration, save the application file and restart the Camelot WCF Service.


Testing the WCF service

One way to test the service if you don't want to start coding right away is through WCFStorm or similar program.

In WCFStorm, make a new connection to the Camelot WCF service.

WCFStorm

When WCFStorm has discovered the service, you should be able to see all the methods on the left side.

WCFStorm

Test the service by selecting one of the methods, for example the ExecuteCamelotXml method, and fill in the method parameters.

sql => For example "select * from Tasks" (selects all columns from the default view of the Tasks list). Please see the official Camelot .NET Connector documentation for all supported SQL commands.

connectionString => The name of the connection string that you wish to use, for example "sharepoint_connection"

compression => Indicates if CamelotXml will be compressed (less data to send over the network), leave this as false (default) for now

sharedKey => Now comes the tricky part, in order to access the service, you must enter the MD5 hash of the secret key. There are several good online tools available that you can use, such as this.

If everything goes fine, when invoking the ExecuteCamelotXml method, you should get a CamelotXml as return value.


Using the toolkit in Visual Studio 2010

Toolkit content description

Name Description Requirements Location
Camelot.SharePointIntegration.dll
  • Utilities for selecting SharePoint data into ListData and DocumentLibraryData objects.
  • Implements the ICamelotService interface that defines the service contract for the Camelot WCF service that is exposed by the Camelot SharePoint Integration Service.
  • Classes and methods used by the Camelot SharePoint Integration Service for automated/scheduled export of SharePoint data.
  • Methods for exporting SharePoint data to MySQL and CSV file formats.
This library depends on the Camelot .NET Connector for querying SharePoint lists, views and document libraries. GAC
Camelot.SharePointIntegration.Client.dll
  • Methods for reading and writing CamelotXml.
  • Defines ListData and DocumentLibraryData classes.
  • String extensions for MD5 and deflate compression.
None. This library allows processing CamelotXml content in client side applications without external dependencies. GAC
Camelot.SharePointIntegration.Plugins.dll
  • Provides a set of standard plugins that can be used in the Camelot SharePoint Integration Service for automated/scheduled export of SharePoint data. Exports include CamelotXmlPush, LibraryToDisk and MySqlDump.

Camelot.SharePointIntegration.dll

Camelot.SharePointIntegration.Client.dll

GAC

CamelotSharePointIntegrationService.exe The "Camelot SharePoint Integration Service" is a standard windows service that hosts the Camelot WCF service and has the ability to execute scheduled exports from SharePoint. It can run on any Windows machine having access to the SharePoint environment(s) via HTTP/HTTPS that you want to integrate with.

Camelot.SharePointIntegration.dll

Camelot.SharePointIntegration.Client.dll

Camelot.SharePointIntegration.Plugins.dll

Installation folder

CamelotSharePointIntegrationService.exe.xml The application configuration for the Camelot SharePoint Integration Service. Modify this to configure Camelot WCF service endpoints and SharePoint connection strings used by the service.

 

Installation folder
Tasks.xml The configuration for automated exports from SharePoint with examples. Modify this to activate exports from SharePoint on certain times.   Installation folder


Adding reference to your VS2010 solution

 In order to use the Camelot SharePoint Integration Toolkit in VS2010 and write your own CamelotXml based applications, you need to add a reference to Camelot.SharePointIntegration.dll and Camelot.SharePointIntegration.Client.dll (which is installed in the GAC).

NOTE! For reading CamelotXml, only a reference to Camelot.SharePointIntegration.Client.dll is required. This library has no external dependencies, thus can be used without Camelot .NET Connector being installed. This is perfect for distributed applications that integrates with SharePoint via the Camelot WCF Service, such as from an external website or in a Umbraco plugin. The Camelot.SharePointIntegration.dll contains utilities that helps selecting data from SharePoint into CameloXml.

  • In VS2010, add a reference to the Camelot.SharePointIntegration.Client.dll and optionally Camelot.SharePointIntegration.dll

VS2010

You should now be able to use the library in your code.


Using the Camelot WCF service

Connecting to the Camelot WCF service in VS2010

To be able to connect to the Camelot WCF service in VS2010 you need:

  1. Reference to the Camelot.SharePointIntegration.Client.dll (installed in GAC)
  2. Service reference to the WCF service installed on any Windows environment that can access SharePoint via HTTP/HTTPS.

In VS2010, add a service reference to the Camelot WCF service.

Example: Querying list "Tasks" via Camelot WCF

The following example shows how to query the Tasks list in SharePoint via the Camelot WCF service. The WCF service returns a CamelotXml package, which is read using the ReadXml method in the CamelotXml class. The resulting ListData object may contain a list description, the list schema and the list items returned by the query. The shared key must match the shared key defined in the WCF service configuration.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Camelot.SharePointIntegration.Client;

namespace ConsoleApplication1
{
    class Program
    {
        const string sharedKey = "MySharedKey";
        static void Main(string[] args)
        {
            var service = new CamelotService.CamelotServiceClient();
            var xml = service.ExecuteCamelotXml(
                "SELECT ID, Title FROM Tasks ORDER BY ID DESC", 
                "sharepoint_connection", false, sharedKey.MD5());

            var listData = CamelotXml.ReadXml(xml);

            foreach (System.Data.DataRow item in listData.Rows.Rows)
            {
                Console.WriteLine(item["ID"] + ", " + item["Title"]);
            }

            Console.ReadKey();
        }
    }
}

Note! Don't forget to import the Camelot.SharePointIntegration.Client namespace to make use of some of the extension methods, such as MD5(), that is defined in the toolkit.

Example: Insert an item into "Tasks" via Camelot WCF

This example shows how simple one can insert items into any SharePoint list. You may use either the ExecuteNonQuery() or the ExecuteScalar() method. The latter returns the ID of the inserted item.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Camelot.SharePointIntegration.Client;

namespace ConsoleApplication1
{
    class Program
    {
        const string sharedKey = "MySharedKey";
        static void Main(string[] args)
        {
            var service = new CamelotService.CamelotServiceClient();
            var result = service.ExecuteNonQuery(
                "INSERT INTO Tasks SET Title = 'My new task', Status = 'Not Started'",
                "sharepoint_connection", sharedKey.MD5());

            if (result)
                Console.WriteLine("Success");
            else
                Console.WriteLine("Failed");

            Console.ReadKey();
        }
    }
}

Connecting to the Camelot WCF service from PHP

The Camelot PHP Tools is an open source (BSD license) project from Bendsoft that simplifies integration with SharePoint via the Camelot WCF service. The following example shows how to query SharePoint using this toolkit. For more details see the documentation for this product.

require_once 'classes/class.camelot.soap.php';

$SharePointQuery = new SharePointQuery(
    array(
        'sql' => "SELECT * FROM `Tasks`",
        'compression' => true,
        'connString' => 'sharepoint_connection',
        'sharedKey' => constant("WSDL_SHARED_KEY")
    )
);

print_r($SharePointQuery);

Integrating your applications with SharePoint

The toolkit provides classes and methods that helps and simplifies SharePoint integration, such as with external applications and websites. The idea behind CamelotXml is to provide a platform independent standard for storing, transferring and processing SharePoint data.

This toolkit offers two general options of integrating with SharePoint.

  1. Through the Camelot WCF service. This is ideal when developing in other languages than .NET and/or when developing "mobile" parts or external applications without direct access to SharePoint. The best about this method is that your application will not require any reference to the Camelot .NET Connector, which allows you to distribute your application easily. You may experience slightly decreased performance compared to direct integration.
  2. Integrate with SharePoint directly using the Camelot .NET Connector and utilities provided by this toolkit.

NOTE! For reading CamelotXml files in .NET, you only need a reference to the Camelot.SharePointIntegration.Client library.

Exporting list data to CamelotXml

You can export data from SharePoint lists and document libraries by providing a specific list and optionally view or by specifying a custom SQL query. With the latter method the resulting ListData object and CamelotXml does not include a list description and the schema is less detailed.

Example: Export information from a specified list and view to CamelotXml

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
	connection.Open();

	var data = Camelot.SharePointIntegration.ListDataUtility.CreateListData("Tasks", "All Tasks", false, connection);
	var xml = CamelotXml.CreateXml(data);

	System.IO.File.WriteAllText(xmlFile, xml, Encoding.UTF8);
}

Example: Export information from the results of SQL query to CamelotXml

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
	connection.Open();

	var data = Camelot.SharePointIntegration.ListDataUtility.CreateListData("SELECT id, title, Status 
FROM tasks where status = 'Not Started' ORDER BY priority"
, connection); var xml = CamelotXml.CreateXml(data); System.IO.File.WriteAllText(xmlFile, xml, Encoding.UTF8); }

Reading CamelotXml content

For reading CamelotXml content in .NET, you only need a reference to the Camelot.SharePointIntegration.Client.dll. The toolkit provides methods for reading CamelotXml into ListData or DocumentLibraryData objects, containing a list description, the list schema and the actual list items. However, the CamelotXml itself can be easily read using any XML reader of your choice. 

Example: Read CamelotXml file into a ListData object and print some information about the content. The ListData class is just one way to read the CamelotXml. You can easily use any other reader/xml parser of your choice, such as the XmlDocument or XmlReader classes.

var xml = System.IO.File.ReadAllText(xmlFile, Encoding.UTF8);
var data = CamelotXml.ReadXml(xml);

if (data.Description != null)
{
    Console.WriteLine("Description");
    Console.WriteLine("  Id: " + data.Description.ID);
    Console.WriteLine("  Title: " + data.Description.Title);
    Console.WriteLine("  Type: " + data.Description.BaseType);
    Console.WriteLine("  Number of items: " + data.Description.ItemCount);
    Console.WriteLine("  Created: " + data.Description.Created);
    Console.WriteLine("  Last modified: " + data.Description.Modified);
    Console.WriteLine();
}

if (data.Schema != null)
{
    Console.WriteLine("Columns");

    foreach (System.Data.DataRow col in data.Schema.Rows)
        Console.WriteLine("  " + col["DisplayName"] + " (" + col["Type"] + ")");
}
else
{
    Console.WriteLine("Columns");

    foreach (System.Data.DataColumn col in data.Rows.Columns)
        Console.WriteLine("  " + col.ColumnName + " (" + col.DataType.Name + ")");
}

Compressed and non-compressed CamelotXml

The toolkit has built-in support for working with deflate compression, which can be useful when transferring and storing CamelotXml files. This is also a built-in feature of the Camelot WCF service and its clients. In .NET you can simply compress (and decompress) using the extended methods defined in the toolkit.

Example: Selecting data from SharePoint and compressing the resulting CamelotXml file before writing to disk.

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
    connection.Open();

    var data = Camelot.SharePointIntegration.ListDataUtility.CreateListData("Tasks", "All Tasks", false, connection);
    var xml = CamelotXml.CreateXml(data);

    System.IO.File.WriteAllText(xmlFile, xml.DeflateCompress(), Encoding.UTF8);
}

Working with document libraries

The toolkit also provides support for working with document libraries, through the DocumentLibraryData and DocumentLibraryNode classes. The latter implements the IHierarchyData interface, making it possible to databind content of a document library to hierarchical data-bound controls, such as the TreeView control.

Example: Export information from a specific document library to CamelotXml

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
    connection.Open();

    var data = Camelot.SharePointIntegration.DocumentLibraryDataUtility.CreateDocumentLibraryListData("Shared Documents", null, connection);
    var xml = CamelotXml.CreateXml(data);

    System.IO.File.WriteAllText(xmlFile, xml, Encoding.UTF8);
}

Example: Print all files in a document library and file size.

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
    connection.Open();

    var data = Camelot.SharePointIntegration.DocumentLibraryDataUtility.CreateDocumentLibraryData("Shared Documents", null, connection);

    foreach (var doc in data.Documents)
    {
        if (doc.ContentType != "Folder")
        {
            Console.WriteLine(doc.Name + " (" + ((double)doc.FileSize / 1024).ToString("N2") + "kB)");
        }
    }
}

Example: Download all files in all folders from document library to disk. It will automatically create all folders. This example also checks last modified date and only updates newer files.

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
    connection.Open();

    var data = Camelot.SharePointIntegration.DocumentLibraryDataUtility.CreateDocumentLibraryData("Shared Documents", null, connection);
    var folder = @"c:\sharepoint_files\";

    foreach (var doc in data.Documents)
    {
        // folder
        if (doc.ContentType == "Folder")
            if (!Directory.Exists(folder + doc.Url))
                Directory.CreateDirectory(folder + doc.Url);

        // document
        if (doc.ContentType != "Folder")
        {
            // skip file if same last modified date
            if (File.Exists(folder + doc.Url))
                if (File.GetLastWriteTime(folder + doc.Url) == doc.Modified)
                    continue;

            using (var command = new Camelot.SharePointConnector.Data.SharePointCommand(
                                                                                string.Format("CALL DOWNLOAD('{0}', '{1}')",
                                                                                "Shared Documents", doc.ProcedureUrl), connection))
            {
                byte[] fileBytes = (byte[])command.ExecuteScalar();
                if (fileBytes != null && fileBytes.Length > 0)
                {
                    File.WriteAllBytes(folder + doc.Url, fileBytes);
                    File.SetCreationTime(folder + doc.Url, doc.Created);
                    File.SetLastWriteTime(folder + doc.Url, doc.Modified);
                }
            }

            Console.WriteLine("Downloaded: " + doc.Name);
        }
    }
}

Example: Databind document library to ASP.NET TreeView control in VB

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
	If Not Page.IsPostBack Then
		Dim connection As New Camelot.SharePointConnector.Data.SharePointConnection(connectionString)
		connection.Open()

		Dim data = Camelot.SharePointIntegration.DocumentLibraryDataUtility.CreateDocumentLibraryData("Shared Documents", Nothing, connection)
		tree.DataSource = data.Documents
		tree.DataBind()
	End If
End Sub

Writing MySQL dumps and CSV files

The toolkit can also be used to export SharePoint content to MySQL via MySQL dump files. These files can be restored into MySQL using the MySQL command tool. It can be very useful when one needs to to more advanced data manipulation and reporting or just for backup.

Example: Export SharePoint list content to MySQL dump file

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
    connection.Open();

    var data = Camelot.SharePointIntegration.ListDataUtility.CreateListData("Tasks", "All Tasks", false, connection);
    var sql = Camelot.SharePointIntegration.MySql.CreateMySqlDump("sharepoint_db", data);

    System.IO.File.WriteAllText(sqlFile, sql, Encoding.UTF8);
}

In similar way, the toolkit provides methods for writing simple CSV files.

Example: Export SharePoint list content to CSV file

using (var connection = new Camelot.SharePointConnector.Data.SharePointConnection(connectionString))
{
    connection.Open();

    var data = Camelot.SharePointIntegration.ListDataUtility.CreateListData("Tasks", "All Tasks", false, connection);
    var csv = Camelot.SharePointIntegration.Csv.CreateCsv(data, ',');

    System.IO.File.WriteAllText(csvFile, csv, Encoding.UTF8);
}
TransportCredentialOnly

Last edited Mar 25, 2013 at 9:03 PM by Bendsoft, version 28

Comments

No comments yet.