Last time, I talked about adding attributes to an entity dynamically. At the same time I was complaining that the code was too complicated, and how I really should be using a transient attribute.

But first, let me recap what I mean by dynamic attributes. They are supposed to behave like normal attributes with the following differences:

• You can add attributes without changing your data model, you can even have attributes that are not known until runtime.
• You can store property lists, including arrays and dictionaries.
• Dynamic attributes can't be used in fetch requests.

The method I am describing here has special handling for multiple managed object contexts writing to the attributes. If you only have a single context, what I am doing here is a bit too much.

Let's see how the old code can be cleaned up.

I am going to move the extendedChanges instance variable into a transient attribute. That should give us automatic undo support and easier handling of -[refreshObject:mergeChanges:YES]. The data model looks like this:

entity MyEntity {
    class ExtendedManagedObject;
    // other attributes...
    binary extendedData;
    transient undefined extendedChanges;
}

I need a single instance variable in my NSManagedObject subclass:

@interface ExtendedManagedObject : NSManagedObject {
    NSDictionary *extendedSnapshot;
}
@end

The snapshot is computed on demand by unarchiving the extendedData plist, exactly like before:

- (NSDictionary*)extendedSnapshot
{
    if (extendedSnapshot!=nil)
        return extendedSnapshot;
    id data = [[self committedValuesForKeys:[NSArray arrayWithObject:@"extendedData"]] 
                objectForKey:@"extendedData"];
    if (data!=nil && data!=[NSNull null]) {
        NSString *error=nil;
        extendedSnapshot = [[NSPropertyListSerialization propertyListFromData:data
                            mutabilityOption:NSPropertyListImmutable 
                            format:nil errorDescription:&error] retain];
        if (error!=nil) {
            NSLog(@"Error loading extended data: %@", error);
            [error release];
        }
    }
    else {
        extendedSnapshot = [[NSDictionary alloc] init];
    }
    return extendedSnapshot;
}

The snapshot represents the value of extendedData in the current database snapshot, so we need to make sure it gets reloaded whenever the snapshot changes. That is in didTurnIntoFault and didSave.

- (void)didTurnIntoFault
{
    [super didTurnIntoFault];
    [extendedSnapshot release];
    extendedSnapshot = nil;
}

Getting the value of an extended attribute is almost the same as before, except for the use of a transient attribute instead of an instance variable:

- (id)valueForExtendedKey:(NSString*)key
{
    id value = [[self valueForKey:@"extendedChanges"] objectForKey:key];
    if (value!=nil)
        return value==[NSNull null] ? nil : [[value retain] autorelease];
    else
        return [[[[self extendedSnapshot] objectForKey:key] retain] autorelease];
}

When changing an attribute, we need to copy the extendedChanges dictionary. This is to ensure that the undo manager does not get confused, as it surely would if we made changes to the contents of a mutable transient attribute.

- (void)setValue:(id)value forExtendedKey:(NSString*)key
{
    NSDictionary *changes = [self valueForKey:@"extendedChanges"];
    NSMutableDictionary *changesCopy;
    if (changes!=nil)
        changesCopy = [changes mutableCopy];
    else
        changesCopy = [[NSMutableDictionary alloc] init];
    if (value==nil)
        value = [NSNull null];
    [changesCopy setObject:value forKey:key]:
    [self setValue:changesCopy forKey:@"extendedChanges"]
    [changesCopy release];
}

We got rid of the messages to the undo manager. Since extendedChanges is a transient attribute, it is handled automatically by Core Data, and undo should work. I have also removed the KVO notifications, not because they were wrong, but they were not sufficient. There are several ways an attribute can change:

• -[setValue:forKey]
• Undo/redo
• -[refreshObject:mergeChanges:]

Putting in -[willChangeValueForKey:] and -[didChangeValueForKey:] only handles KVO notifications for the first case. To get at the other two cases, we must tell Core Data that the new attribute depends on extendedChanges and extendedData:

+ (void)initialize
{
    ...
    [self setKeys:[NSArray arrayWithObjects:@"extendedChanges", @"extendedData", nil]   
        triggerChangeNotificationsForDependentKey:@"myAttrName"];
}

The class message must be sent before any instances are created, so this method won't work for keys that are not known at compile time. You can still use runtime-defined keys, you just won't get KVO notifications for them.

Saving works just like before:

- (void)willSave
{
    NSDictionary *changes = [self valueForKey:@"extendedChanges"];
    if (changes!=nil) {
        // merge changes into snapshot
        NSMutableDictionary *dict = [[self extendedSnapshot] mutableCopy];
        NSEnumerator *e = [changes keyEnumerator];
        NSString *key;
        while (key=[e nextObject]) {
            id value = [changes objectForKey:key];
            if (value==[NSNull null])
                [dict removeObjectForKey:key];
            else
                [dict setObject:value forKey:key];
        }

        // archive as binary plist
        NSData *data = nil;
        if ([dict count]>0) {
            NSString *error=nil;
            data = [NSPropertyListSerialization dataFromPropertyList:dict 
                    format:NSPropertyListBinaryFormat_v1_0 errorDescription:&error];
            if (error!=nil) {
                NSLog(@"Error saving extended data: %@", error);
                [error release];
            }
        }
        [dict release];
        [self setPrimitiveValue:data forKey:EXTDATAKEY];
    }
    [super willSave];
}

In didSave, remember to clear both extendedChanges and extendedSnapshot:

- (void)didSave
{
    [super didSave];
    [extendedSnapshot release];
    extendedSnapshot = nil;
    [self setPrimitiveValue:nil forKey:@"extendedChanges"];
}

That is all. We no longer need to subclass NSManagedObjectContext, and undo is handled automatically. The price we pay is copying the extendedChanges dictionary whenever an extended attribute is changed. It is just a shallow copy, though, so it's not so bad.

Accessors

There are two ways of writing accessors for extended attributes. One is to use the general KVC fallback mechanism:

- (id)valueForUndefinedKey:(NSString*)key
{
    return [self valueForExtendedKey:key];
}

That way, our class is suddenly KVC compliant for any key. This is a bit dangerous because a misspelled key just becomes a new attribute. You won't get any warnings, just values disappearing mysteriously. A safer approach is to write a named accessor method:

- (id)myAttrName
{
    return [self valueForExtendedKey:@"myAttrName"];
}

Finally, for runtime-defined keys, you can just call -[valueForExtendedKey:] directly. Remember that KVO depends on -[valueForKey:], so if you enable KVO for a key, you must make sure that the class is KVC compliant for that key.

How it works

When an extended managed object is first inserted, extendedSnapshot is empty and extendedChanges is nil, so -[valueForExtendedKey:] returns nil for any key. Whenever you set an extended attribute, a copy of extendedChanges is made with the new value and stored in the transient attribute.

Saving the managed object context merges the changes into extendedSnapshot, and that is archived as a binary plist in the extendedData attribute. Note that willSave changes neither extendedSnapshot nor extendedChanges. If the save fails for some reason, we want to still keep track of the changes, therefore extendedSnapshot and extendedChanges are not cleared until didSave. That is also the reason -[extendedSnapshot] uses -[committedValuesForKeys:] raher than the simpler -[valueForKey:] when retrieving the archived plist. A failed save followed by a refresh would otherwise cause us to use the wrong data for the snapshot.

Refreshing the object works. [refreshObject:mergeChanges:NO] clears all transient attributes, including our changes. [refreshObject:mergeChanges:YES] will clear extendedSnapshot in didTurnIntoFault, but preserve extendedChanges. Next time we access an attribute, extendedSnapshot is reloaded, including any changes made by another context. This means that attributes we haven't touched are refreshed, while the attributes we changed are preserved. Note that the semantics here are different from standard attributes. Core Data will compare values when determining if an attribute is changed, but extended attributes use the "-[setValue:forKey:] called" criteria. The difference is subtle.

Undo and redo works too. Transient attributes are reverted to previous values, including the extendedChanges dictionary. All our changes are reverted.

Saving

I forgot to mention this last time. If you have multiple contexts altering your extended attributes, you need special code for saving. Suppose one context changes the attribute "foo" while another context changes "bar". Now they both want to save. For normal attributes, you can handle this by setting a NSMergeByPropertyObjectTrumpMergePolicy, but that won't work for extended attributes. They are all stored in the same binary attribute, and of course Core Data has no way of knowing how to merge them properly. If you use NSMergeByPropertyObjectTrumpMergePolicy, it effectively becomes NSOverwriteMergePolicy for extended attributes. Not good.

To fix this, we set an NSErrorMergePolicy (the default) and merge conflicting objects manually:

NSError *error = nil;
while (![moc save:&error]) {
    if ([[error domain] isEqualToString:NSCocoaErrorDomain] 
            && [error code]==NSManagedObjectMergeError) {
        NSArray *conflictList = [[error userInfo] objectForKey:@"conflictList"];
        NSEnumerator *e = [conflictList objectEnumerator];
        NSDictionary *conflict;
        while (conflict=[e nextObject]) {
            [self refreshObject:[conflict objectForKey:@"object"] mergeChanges:YES];
        }
    }
    else {
        // handle other error (typically validation)...
        break;
    }        
}

When a save fails because of merging conflicts, we send -[refreshObject:mergeChanges:YES] to all the conflicting objects. This behaves like an NSMergeByPropertyObjectTrumpMergePolicy for normal attributes, and it causes extendedSnapshot to be updated with the changes from the persistent store, so when we try to save again, -[willSave] will merge the right values. The while loop is necessary because a third context may save while we are merging the chanages from context number two. If you are rapidly changing and saving a context, the loop could run forever, so don't do that.

This way of saving changes the semantics a bit. Some of your objects may be refreshed; that never happens with a 'normal' save. It is only necessary when you have multiple contexts editing your extended attributes, but if you don't have that, you don't need the fancy two-layered attribute storage either. If you save normally, using NSMergeByPropertyObjectTrumpMergePolicy, all your extended attributes will behave like a single attribute w.r.t. merging. The last context to save overwrites all extended attributes to whatever it thinks they should be. (Usually their values the last time the context was saved.)

Finally, let me say that I think it is really cool that Core Data lets you do stuff like this. The Cocoa design patterns that let you extend the functionality of system frameworks are awesome. Great job, Core Data team!