WCF -> JSON Serialization woes and a solution

I’m working on improving the performance of my current web application project at various points. As I had already planned for a WCF interface for third-party use, I thought I’d utilise that, exposing objects as JSON-serialized strings usable by jQuery/ASP.NET AJAX.

Little did I realise that when I was coming up with my Data Framework (I chose Entity Framework) I should have thought about aspects at the other side of my project such as the User Interface as well as the more obvious aspects of scalability, performance in relation to a pragmatist view of what is needed in the “real world” (therefore, nHibernate is out).

My WCF service had an exposed service that would return a Client object (actually, an interface IClient) and return it as JSON. Simple.

No, I hit a number of issues with this.

1/ Investigations at StackOverflow indicated that Interfaces cannot be exposed through serialization. Makes sense, really, but dents my idealistic view of presenting interfaces and no conncrete objects to third-parties via my API. So I was going to have to either return my concrete object or reduce my return value to a JSON string, thereby bringing serialization “in house” and not relying on WCF to serialize it for me.

From:

[OperationContract]
[WebInvoke(Method="POST",BodyStyle=WebMessageBodyStyle.Wrapped,ResponseFormat=WebMessageFormat.Json)]
IClient GetClientJson(int clientId);
I went to:
[OperationContract]
[WebInvoke(Method="POST",BodyStyle=WebMessageBodyStyle.Wrapped,ResponseFormat=WebMessageFormat.Json)]
string GetClientJson(int clientId);

2/ This actually dodged the issue of the attribute configuration on the service itself, which was becoming a nightmare. Tweak it at the server and it breaks at the client, and vice versa. What was:

From:

[OperationContract]
[WebInvoke(Method="POST",BodyStyle=WebMessageBodyStyle.Wrapped,ResponseFormat=WebMessageFormat.Json)]
string GetClientJson(int clientId);
I went to:
[OperationContract]
[WebInvoke(Method="POST",BodyStyle=WebMessageBodyStyle.Bare,ResponseFormat=WebMessageFormat.Json)]
string GetClientJson(int clientId);

3/ Next was serializing the object into JSON. I was getting the “The type ‘xxx’ cannot be serialized to JSON because its IsReference setting is ‘True’. The JSON format does not support references because there is no standardized format for representing references. To enable serialization, disable the IsReference setting on the type or an appropriate parent class of the type.” exception. As I was essentially getting this at the client, it suggested my WCF endpoint configuration was wrong. Looking deeper, it turns out that Entity Framework objects are marked with IsReference=True, meaning the native DataContractJsonSerializer of WCF cannot serialize Entity Framework objects. I proved this by doing a manual serialization:

string jsonClient;
IClient client = GetClient(7);
DataContractJsonSerializer ser = new DataContractJsonSerializer(client.GetType());
using (MemoryStream ms = new MemoryStream())
{
    ser.WriteObject(ms, client);
    jsonClient = Encoding.Default.GetString(ms.ToArray());
}
return jsonClient;

4/ I needed to serialize using a different serializer, so thought I’d use the ASP.NET AJAX Serializer, which also didn’t work, this time falling over the exception “A circular reference was detected while serializing an object of type xxx’.” The type it was complaining about wasn’t in the object so it was clearly navigating deeper into other objects to find that particular Type anyway.

5/ So now I am left with no other option but to either do it myself or use a third-party library. I’m using Json.NET, which I spotted on Scott Hanselman’s blog and seems to be robust enough and simple enough for most purposes. So my code now looks like:

string jsonClient=null;
IClient client=GetClient(1);
JsonSerializer jsonSerializer = new JsonSerializer();
jsonSerializer.Converters.Add(new JavaScriptDateTimeConverter());
jsonSerializer.NullValueHandling = NullValueHandling.Ignore;
jsonSerializer.MissingMemberHandling = MissingMemberHandling.Error;
jsonSerializer.ReferenceLoopHandling = ReferenceLoopHandling.Error;
try
{
     using (StringWriter sw = new StringWriter())
     {
          using (JsonTextWriter jtw = new JsonTextWriter(sw))
          {
                 jsonSerializer.Serialize(jtw, client);
          }
     }
}
catch (Exception ex)
{
       ex = ex; // have a breakpoint here so can inspect exception
}
return jsonClient;

Notice I have set ReferenceLoopHandling to ReferenceLoopHandler.Error. This is to try and catch the same Reference Count issue that ASP.NET AJAX JSON Serialization catches. (Actually this was added after realising I had StackOverflows occurring). Sure enough, I have another Reference Count issue as the Exception does get caught and the error is related to possible infinite recursion.

The JSON.NET Framework allows me to disable serializing potentially problematic objects, but this would require applying these changes to elements of code “Bhind the wall” of my API – and essentially add Web-specific functionality into a domain that is supposed to be platform agnostic. This is not an option for me.

So I appear to be stuck. Other than rendering the JSON myself through a StringBuilder, I’m pretty much stuck on this now. Maybe something will hit me in a flash of inspiration. Until then, it’s good ol’ StringBuilder for me.

Update: The Solution:

Thanks again due to StackOverflow, (John Saunders and Craig Stuntz) I’ve figured out how I’m going to do it. It’s not as pretty as I would have liked, but pragmatism wins out again.

Here it is from start to finish. My UI generates an event that is picked up by some JavaScript. This runs:

var wcfProxy = new serviceProxy("../api/wcf/ClientBroker.svc/");
wcfProxy.invoke("GetClientJson", { clientId: 7 }, updateClient, updateClientError);

serviceProxy is RickStrahl’s WCF wrapper, which has two callbacks, success and error (the last two parameters, respectively). The Invoke method invokes the WCF service and obtains the result. The WCF service is exposed via the Interface:

[ServiceContract(Namespace = "xxxWCF")]
public interface IClientBroker
{
     [OperationContract]
     [WebInvoke(Method="POST",BodyStyle=WebMessageBodyStyle.Wrapped,ResponseFormat=WebMessageFormat.Json)]
     string GetClientJson(int clientId);
}

Note that the Body STyle is Wrapped. I couldn’t get it working in Bare mode at all. (Despite telling me it had logged messages in the Windows Event Log, no events were to be found).

[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
public class ClientBroker : IClientBroker 
{
     public string GetClientJson(int clientId)
     {
            string jsonClient=null;
            IClient client = GetClient(clientId);
            var j = new { ID=client.ID, BusinessName=client.BusinessName };
            JavaScriptSerializer s = new JavaScriptSerializer();
             jsonClient=s.Serialize(j);
      }
      return jsonClient;
}

Notice I am using Anonymous Types to create a new type which is free of Entity Framework idiosyncracies.

This generates a wrapped JSON string:

{"GetClientJsonResult":"{\"ID\":7,\"BusinessName\":\"XYZ Ltd\"}"}

When returned to the client, the success callback is called:

function updateClient(o) {
         eval('var z=' + o');
         alert('BusinessName=' + o.BusinessName);
}

And the Business Name of the requested client is displayed.

It just goes to show that Microsoft may put all the features they like into C#, and you might very well know they are there. But you’re not going to use them until you need to use them, and then you need to know that you need to use them! Variant types are, to me, an uncomfortable throwback of those bad VB days but sometimes they can prove useful when it comes to the crunch, I’m just keen on restricting my use of them to the absolute minimum so the very premise of a type-strong language is not lost.

Another stone passed.

 

Facebook Home Page

I’m not going to be the first and I won’t be the last to write about this, so I won’t labour the point. Hopefully the graphic says it all. The new Facebook home page seems to be trying to be more like Twitter than actually bringing prople together, which it was previously very good at. With the posted items and “ancillary” activity being demoted to the right column, I am now no longer as immersed in what my friends are posting, doing or playing. Some of my friends posted articles of interest, which allowed me to comment there and then. Now I only see abbreviations. The core area is now trying to be a Twitter copy, being filled with everyone’s status updates … oh, sorry “whatever is on their mind”.

Facebook home page

Verdict: Meh.

Jacqui Smith: “I know, we’ll create a huge database”

The UK is facing a number of challenges that span professional bodies and
disciplines, which with good intentions, the Home Secretary Jacqui Smith is sure
she can solve.

Helping prevent child abuse

Child abuse is a horrendous tragedy that has recently seen a number of nasty
headlines. The Baby P scandal was a disturbing case that exposed many weaknesses in the various professional bodies that come into contact with families regarded as being “at risk”.

Jacqui Smith: “I know, we’ll create a huge database”

The joining together of separate databases held by the police, the courts, social
workers, teachers and the health service to create a “super database” containing
details on 11 million children under 18 years of age is believed to be illegal,
falling foul of the Data Protection Act. Projected cost: £200 million.

CCTV CamerasHelping reduce domestic violence

Domestic violence is the invisible abuse that partners suffer in silence.
There can be little worse betrayal than being abused by the one you love, and it
can prove very frightening – often so frightening that it becomes almost too
terrifying to seek help.

Jacqui Smith: “I know, we’ll create a huge database”

The generation of a database of perpetrators has been criticised as a gimmick
and caused Jacqui Smith to be publicy shouted down by activists. There are a significant number of people wondering why more effort is not put in addressing weaknesses in police powers who feel powerless in such situations.

Solving crimes retrospectively using advances in DNA

Some horrendous crimes have been solved thanks to the advances made in DNA
analysis, which provides convincing scientific proof of a suspect’s guilt. As
technology develops, and our understanding grows, we need to be able to utilise
our increased knowledge to help solve past crimes.

Jacqui Smith: “I know, we’ll create a huge database”

The process of having your fingerprints taken at the police station on arrest
is well known. Until recently, merely being arrested required you to submit a
DNA sample for entering into the National DNA database. If you were later released, your sample was not, instead remaining “on file” – just in case.

Modern communications may be used to organise terrorist acts

Clearly, terrorism remains a potent risk to the society, and it has been seen
that our intelligence services have struggled to keep up with the technological
advances brought by communications technologies such as mobile phones and the
internet.

Jacqui Smith: “I know, we’ll create a huge database”

Forcing ISPs, telecommunications operators and related services to provide
details of communications to the government for recording in a huge central
database
will allow government to be able to access details of communications and trace activity regarded as suspicious. I’m sure their IT systems would be up to the task of indexing such a massive database.

Tracking people coming in an out of the country reduces terrorism/illegal immigration

The huge number of people coming into the country and leaving the country
presents immigration officials and security services  problems in identifying
individuals who may pose a risk to the United Kingdom or one of its
neighbours.

Jacqui Smith: “I know, we’ll create a huge database”

Going through a UK airport will now see you photographed and monitored from
booking to plane, with this information potentially being shared across country
boundaries to your intended destination. The project, known as “Semaphore
will store passenger movements within a central database, even those of 6
year-olds
.

Keeping track of civilians will help prevent terrorist attacks

The globalisation of the economy and increased migration of individuals
increases the United Kingdom’s exposure to terror attacks such as September11th
or the attacks on Madrid.

Jacqui Smith: “I know, we’ll create a huge database”

Creating a new National Identity Card system, over and above the existing Passport obligations of civilians, will help authorities prevent individuals intent on attacking the United Kingdom and entering it illegally. EVen though Spain already has an ID card system, this did not prevent the Madrid attacks.

I wonder if Jacqui Smith realises she is a member of the same government that
is responsible for the disastorous NHSPfIT programme designed to streamline the
IT services of the NHS, the crashing of a number of systems, massive overspend
and terrible planning of a number of high-profile IT projects. I wonder if she can recall the extensive “misplacing” of data that occured in 2008, also under her government. I wonder what makes her think that more databases, more lines crossed in people’s liberties and an increased feeling of distrust can do to help her government remain a tenable prospect for 2010 onwards. No, she is just as misguided as the Prime Minister that went to an illegal war and the “other” Prime Minister who can’t say “sorry”. Her one and only solution is a reflection of the ineptidude of the government’s IT policy. Hopefully she’ll fall over before it becomes too late for a replacement government to be able to reverse such damaging changes to our constitition.