Design Meeting Notes - November 8, 2012

Lightweight conventions

After playing with the lightweight convention APIs there was a feeling that the APIs could be made:

  • Cleaner/simpler
  • Most consistent with existing APIs/patterns
  • Provide better support for DRYer code in common cases

Specific changes:

  • One of the nested closures can be removed if the lightweight APIs follow more of the fluent/builder pattern rather than always creating something that is added to the conventions list. To do this, there needs to be two entry methods: one for entities and one for properties. One consequence of this is that putting these methods on Conventions, which follows the collection pattern in a similar way to Configurations, would be mixing the two patterns. Therefore, the plan now is to move back to hanging these methods directly off the model builder. For example:
    modelBuilder.Entities()
        .Configure(c => c.HasKey(“Key”));
    modelBuilder.Properties()
        .Configure(c => c.HasPrecision(5, 5));
    
    • This now makes it not completely obvious that these are conventions and not configurations. We could add the word “conventions” to the method names to make this clearer, but we don’t believe that the distinction is relevant for the vast majority of uses. In cases where it may be relevant (such as the fact that calling Entities doesn’t add entities to the model, and also, possibly, ordering) it is relatively easy to discover from Intellisense and documentation what the behavior is and how it works.
  • Allowing generic versions of these methods that accept base types or interfaces make the experience much better for cases where such as base type or interface exists. This is very similar to the ChangeTracker.Entries case. For example:
    modelBuilder.Entities<IEntity>()
        .Configure(c => c.HasKey(x => x.Key));
    modelBuilder.Properties<decimal>()
        .Configure(c => c.HasPrecision(5, 5)); 
    
  • The current use of properties inside the configuration should be changed to use methods for several reasons:
    • When the value of the property is read it is not at all clear what this value means. Specifically, it is the value that has been configured, if any. If no value has been configured, then the value is null and not the default value that will ultimately make it into the model. If you want to read the model and perform operations based on this then a model convention is the way to do it.
    • The methods match the existing well-known methods that are already used in the existing fluent API.
    • Chaining of methods can now happen in the same way as it does for the existing fluent API. For the generic entities method this chaining can also be restricted based on type, just as in the existing fluent API. For places where there is not enough information to restrict the chaining is flattened just as it is at the top level. Chaining of methods now means that the overload of IsKey that takes a column order is not really needed—you can just chain a call to HasColumnOrder.
  • The idea of being able to drop down to properties (that was discussed previously) should be included. This allows a convention to filter by entity and then easily do configuration of its properties. For example:
    modelBuilder.Entities<IEntity>()
        .Where(e => e.Name.EndsWith("Foo"))
        .Configure(c => c.Property(e => e.Bar).HasPrecision(5, 5)); 
    
  • It seems reasonably common that the results of filtering in a Where will also be needed in the Configure method. Rather than having to write this code twice or factor it out we are introducing a Having method. For example:
    modelBuilder.Properties<decimal>()
        .Having(p => p.CustomAttributes.OfType<DecimalPrecisionAttribute>().FirstOrDefault())
        .Configure((c, a) => c.HasPrecision(a.Precision, a.Scale));
    
    • Having does two things:
      • If the lambda returns null, then the entity or property is filtered out in the same way as it would be if the predicate in Where returned false.
      • If the lambda returns non-null then application of the convention continues with the non-null return value being passed to the Configure method. This means that the Configure method after a Having takes a two argument lambda.
    • Having could allow Where after the Having or it may not. It doesn’t seem super-compelling, but it also doesn’t seem compelling to leave it off.
    • When now becomes Where again to fall in line with the Having pattern. It is possible to have a dangling Where or Having; we will either throw or do nothing for these cases.
    • We considered special casing IEnumerables in Having such that returning an empty enumeration is equivalent to returning null, but this adds complexity without much value.

Connection resiliency to transient failures

Goals

  • Good out-of-box experience with SQL Azure
    • Mechanism is pluggable and can be associated with providers to allow it to be used in other situations

Non-Goals

  • Fully supporting existing EF applications without changes being made to those applications
    • Resiliency is off by default, although we will try to help people know about it by proving information in transient failure exceptions

Scope

  • DbContext API (SaveChanges, queries, possibly commands)
  • ObjectContext API as needed
    • Often implementation will be at the ObjectContext layer anyway
  • Ability to apply the retry policy on other methods manually

Assumptions

  • SQL Azure can drop connections due to:
    • Excessive resource usage
    • Long-running transactions
    • Failover and load balancing actions
    • Federations rebalancing
    • Network connectivity issues
  • Transaction issues:
    • SQL Azure does not support distributed transactions
    • SqlClient does not support nested nor parallel transactions
    • Transient failures cause connection to close and the transaction to rollback

Proposal

  • SaveChanges
    • Create and use one local transaction for all commands as there is no way of dividing them in retryable chunks while keeping the ability to rollback all of them if a non-transient error is encountered
  • Queries
    • Buffer the results (at DbDataReader level) before returning them if retry policy is enabled.
      • We will likely switch to buffering by default due to other benefits
      • Buffering at the data reader level keeps the state manager state consistent even if there is a retry
    • Allow to deactivate the retry policy (and thus buffering) for some queries as they could be too large to fit in the memory (For example, AsStreaming method)
  • Raw SQL execution and other methods
    • Do use the retry policy
    • Always enabled for EntityConnection Open and BeginTransaction
  • Policy behavior
    • Presented as a provider service
    • Can be changed using Service Locator
    • User can get the current policy for the current provider
    • Only retries on transient errors, can be overridden
    • Throws when enabled and Transaction.Current is not null; may be possible to override this
    • Can configure streaming/buffering on a per-query basis
    • Algorithm for determining the delay between retries can be overridden
    • If retry limit reached throws an exception suggesting to break operations into smaller transactions,  this can be turned off
    • Don't provide a Retrying event to keep RetryPolicy immutable. Tracing feature should be enough to cover most use cases. Alternatively we could add a callback as a constructor parameter.
  • SQL Azure specific retry policy
    • By default is not enabled
    • When enabled will retry for SaveChanges, tracking queries and notracking queries, but they can be opted-out individually.
    • Exponential wait period increasing algorithm with a small random factor. First delay is close to 0.
    • 5 retry limit ( about 1 minute delay in total), consider 15 secs

Open Issues

  • Should the default streaming be at the data reader level or at the ObjectResult level?
    • Start with data reader level; this will use more memory but means buffering is only used at one place and behavior is more consistent with current behavior
  • Should we wrap transient exceptions with a more helpful message if SQL Azure retry policy isn't enabled?
    • We should try this, making sure to keep the inner exception the same for people currently using bespoke retry code
  • Possible breaking change (Different merge options, interrupted streaming, etc.)
  • Should NoTracking, SqlQuery (DbSet version and/or Database version) and SqlCommand methods use the policy?
    • Yes for all queries
    • For commands more work is needed to understand the implications of introducing a transaction

Last edited Dec 14, 2012 at 6:08 PM by ajcvickers, version 2

Comments

No comments yet.