-
Notifications
You must be signed in to change notification settings - Fork 934
NH-2319 - Add ability to query collections with AsQueryable() #375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason for not including idbags (PersistentIdentifierBag
) and lists (PersistentGenericList
) in the feature? Reading now the Jira, is this PR still limited to bidi one-to-many bag and set?
A bit like Fabio, I am not much convinced by the feature. It is nice to avoid loading the full collection in some cases, but well, this can easily be achieved with a query instead. Of course it is less straight forward, but since the feature seems currently so limited, it still looks as the default way to go for me.
Otherwise, I think we need an obvious mean for knowing we are in a currently unsupported case: throw an exception instead of falling back on linq-to-object? This has the advantage of being obvious for the developer. While the fallback to linq-to-object triggers silently the full load of the collection (and I guess from batch-size others too) and may be unnoticed by the developer. But if anyone has put some pointless AsQueryable()
in its current code, it will be a breaking change. Maybe add a configuration property for choosing between fallback or throw.
For supporting maps, we would have to provide a dictionary with a custom Values
implementation. Maybe not worth the trouble.
My review is just preliminary, I have not thoroughly studied (and understood) how this new feature works.
@@ -46,6 +49,8 @@ public class PersistentGenericBag<T> : AbstractPersistentCollection, IList<T>, I | |||
*/ | |||
private IList<T> _gbag; | |||
|
|||
private IQueryable<T> queryable; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tend to use _
underscore prefix for private fields, and changes fields which does not have it when I have already change a lot in the class. I see that these new queryable
fields do not have it. Are we accepting both, or shall we use a single style, and which one?
I think this field needs the [NonSerialized]
attribute.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a hundreds years old PR, so I don't remember/know why I didn't put underscore in front of the field name.
Agreed on [NonSerialized]
part.
|
||
private IQueryable<T> InnerQueryable | ||
{ | ||
get { return queryable ?? (queryable = new NhQueryable<T>(Session, this)); } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't that the job of a new Lazy<NhQueryable<T>>(() => new NhQueryable<T>(Session, this))
which would be affected to a retyped queryable
in the constructor?
I know we do not need thread safety here. This is just for avoiding coding this additional property and having a field which can be null if a later coder access it instead of using InnerQueryable
.
Of course, we would then have to replace InnerQueryable
calls by queryable.Value
.
Other consideration: I think the field must be marked as non serializable. Better having them lazily recreated after deserialization. (The current pattern better support it. With a Lazy, we would need to implement ISerializable
to recreate it I fear.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lazy<>
is not really required. What's required is a proper implementation of IQueryable<>
, and it turns out that NhQueryable<>
is doing almost everything what we need.
@@ -38,6 +40,8 @@ public class PersistentGenericSet<T> : AbstractPersistentCollection, ISet<T> | |||
[NonSerialized] | |||
private IList<T> _tempList; | |||
|
|||
private IQueryable<T> queryable; | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same remarks as for previous class.
|
||
if (plan == null) | ||
{ | ||
if (log.IsDebugEnabled) | ||
{ | ||
log.Debug("unable to locate collection-filter query plan in cache; generating (" + collectionRole + " : " | ||
+ filterString + ")"); | ||
log.Debug(string.Format("unable to locate collection-filter query plan in cache; generating ({0} : {1})", collectionRole, queryExpression.Key)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess at the time of this change, $"" formatting was not available!
I am frequently a bit surprised by those if (log.IsDebugEnabled)
: they are pointless if they are just about avoiding formatting without any value evaluation, aren't they? log.DebugFormat("", args)
already handle that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess at the time of this change, $"" formatting was not available!
True. However, It seems that it shall be changed to log.DebugFormat(..)
if (log.IsDebugEnabled)
It seems just to avoid doing string concatenations and win some milliseconds and object allocations (in some cases it's actually a huge win)
Haven't found anything on C#, so the links are about Java, but internals are almost the same, so
- http://stackoverflow.com/questions/963492/in-log4j-does-checking-isdebugenabled-before-logging-improve-performance
- https://www.igorkromin.net/index.php/2015/05/12/are-guard-statements-like-isdebugenabled-necessary-when-using-log4j/
- http://learnertobeginner.blogspot.co.nz/2010/06/how-to-use-isdebugenabled-effectly.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am used to rarely need the explicit check, because I am mostly using Format
suffixed methods when having to format a log. Of course sometime we need to format with computed values, or to have a more complex message construction. Then I use the explicit check. I am sensible to this because log messages are almost bloat code in my view, and I rather keep them as compact as possible. (Actually I tend to only log exceptions. I fully agree with Jeff Atwood blog on this.)
|
||
|
||
if (fromElementFound) | ||
return (IASTNode) adaptor.Nil(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Has this been done because some of the calls above would have side effects? If yes, maybe a comment would be welcome. Otherwise I would tend to move this return
in the if (fromElement == null)
clause.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I understand after a quick look-around. There are some differences in the AST between HQL and LINQ. It seems that LINQ AST already contains fromElement
in the tree, and HQL does not.
We still need to do all these manipulations with the fromElement
, but we do not want to add it back to the tree (because it is already there).
src/NHibernate/Impl/SessionImpl.cs
Outdated
CheckAndUpdateSessionStatus(); | ||
queryParameters.ValidateParameters(); | ||
var plan = GetFilterQueryPlan(collection, queryExpression, queryParameters, false); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost dup code here. I think this need to be refactored with above method.
src/NHibernate/Impl/SessionImpl.cs
Outdated
} | ||
} | ||
|
||
private IQueryExpressionPlan GetFilterQueryPlan(object collection, string filter, QueryParameters parameters, bool shallow) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here, lot of dup code with above method, should probably be refactored.
src/NHibernate/Linq/NhQueryable.cs
Outdated
@@ -46,11 +46,17 @@ public NhQueryable(IQueryProvider provider, Expression expression, string entity | |||
EntityName = entityName; | |||
} | |||
|
|||
public NhQueryable(ISessionImplementor session, object collection) | |||
: base(new DefaultQueryProvider(session, collection)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't it use the QueryProviderFactory
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. But it wasn't here when this code was written.
It's because this feature is really just a strongly-typed wrapper on a filter collection feature which does not support these collections |
About the differences in LINQ vs HQL: Linq generates following AST:
And HQL:
So, it's better to eliminate the initial difference in the AST tree instead of modifying CreateFromFilterElement |
I never needed to use filters so I am not sure, but are filter supporting batch loading of collection? I tend to think yes, and then, deprecating filters for queryable collections may cause a significant performances loss, unless we find a way to maintain batching with queryable collections. |
Hibernate team seems to want to only remove the special query on collection case. I was thinking about the more global filters functionality, declarable in mappings. Well, I never found this query on collection feature very useful, it is so easy to just write a usual query doing the same job. |
Yes, only this
Not these filters:) I know it confusing to have 2 distinct features share similar names
Actually, [some] people are desperate to have this feature implemented. It's one of the highest voted features in the JIRA. |
Later |
c9975fb
to
cb1930e
Compare
Rebased. Once rebased some tests were failing when run in sequence with others, due to a trouble when a filtered query plan was retrieved from cache. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some monthes ago I had not checked the Jira proposed implementation.
Appearing more simple, it has probably way more limitations and shortcomings than the one you have made.
Maybe we should add more tests for checking that your implementation support all non bidi cases, perhaps many-to-many too, ... All cases supported by the filter feature should be supported, but I do not know much what it does support in fact.
I forgot to regen async for all commits, going to do that too.
Edit: just realized idBag were many-to-many. So it just lacking testing other unidi cases and maybe bidi with inverse.
@@ -79,7 +79,7 @@ public void Reset() | |||
} | |||
} | |||
} | |||
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cleanup glitch, going to fix and repush forced.
src/NHibernate/Impl/SessionImpl.cs
Outdated
role => Factory.QueryPlanCache.GetFilterQueryPlan(filter, role, shallow, EnabledFilters)); | ||
} | ||
|
||
private IQueryExpressionPlan GetFilterQueryPlan(object collection, QueryParameters parameters, Func<string, IQueryExpressionPlan> getPlanForRole) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I usually avoid this kind of pattern, putting code in common thanks to function pointers passed around. But here, with the function being defined for all cases right above, I think it is ok.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does not seem compatible with Async generator, causes the removal of GetFilterQueryPlanAsync
. Now I wonder what IO are there in that function, going to check. Ok it flushes, already forgot...
// For backward compatibility, prioritize using the version without collection. | ||
return (collection == null | ||
? Activator.CreateInstance(session.Factory.Settings.LinqQueryProviderType, session) | ||
: Activator.CreateInstance(session.Factory.Settings.LinqQueryProviderType, session, collection)) as INhQueryProvider; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minimal backward compatibility handling.
cb1930e
to
9ac4f3c
Compare
if (filter != null && queryExpression != null) | ||
throw new ArgumentException($"Either {nameof(filter)} or {nameof(queryExpression)} must be specified, not both."); | ||
if (filter == null && queryExpression == null) | ||
throw new ArgumentException($"{nameof(filter)} and {nameof(queryExpression)} were both null."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not like this kind of parameter semantic but do not see a better solution for avoiding code duplication and not using a pattern not handled by async generator.
0c48f88
to
642979f
Compare
@@ -73,6 +74,7 @@ public TextLogSpy(string loggerName, string pattern) | |||
}; | |||
loggerImpl = (Logger)LogManager.GetLogger(typeof(LogsFixture).Assembly, loggerName).Logger; | |||
loggerImpl.AddAppender(appender); | |||
previousLevel = loggerImpl.Level; | |||
loggerImpl.Level = Level.All; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This spy was used only for NHibernate.SQL
, but that is an heavily logging logger. Better restore its previous log level.
@@ -684,7 +684,6 @@ void Clone_TransactionCompleted(object sender, TransactionEventArgs e) | |||
[OneTimeSetUp] | |||
public void TestFixtureSetUp() | |||
{ | |||
((Logger)_log.Logger).Level = log4net.Core.Level.Info; | |||
_spy = new LogSpy(_log); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Undue log level manual change, since LogSpy
set it to Debug
anyway.
642979f
to
b61a5a9
Compare
92f3859
to
36f5cb1
Compare
doc/reference/modules/query_linq.xml
Outdated
<para> | ||
If the collection is a map, call <literal>AsQueryable</literal> on its <literal>Values</literal> | ||
property. | ||
</para> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added support for map.
Assert.That((ICollection) country.Statistics.Keys, Is.EquivalentTo((ICollection) stats.Keys), "Keys"); | ||
Assert.That((ICollection) country.Statistics.Values, Is.EquivalentTo((ICollection) stats.Values), "Elements"); | ||
Assert.That(country.Statistics.Keys, Is.EquivalentTo(stats.Keys), "Keys"); | ||
Assert.That(country.Statistics.Values, Is.EquivalentTo(stats.Values), "Elements"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those casts were likely required for an older NUnit. Since I have not additionally implemented ICollection
on the map values wrapper, they were failing. We will probably have to document that as a possible breaking change.
var filtered = parent.ChildrenMap.Values | ||
.AsQueryable() | ||
.Where(x => x.Name == "Piter") | ||
.SelectMany(x => x.GrandChildren) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a map here. Supporting maps inside lambda would be another feature. (Currently already doable by pretending the map is a ICollection<TValue>
with a cast, but a bit hackish.)
rc.Id(x => x.Id, m => m.Generator(Generators.GuidComb)); | ||
rc.Property(x => x.Name); | ||
rc.Map(x => x.ChildrenMap, | ||
map => map.Cascade(Mapping.ByCode.Cascade.All | Mapping.ByCode.Cascade.DeleteOrphans), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have not found how to specify the index, although we can specify it in hbm. Missing feature of by code mapping? Anyway, works with defaults for the test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is supposed that you use the Map
overload which takes 4 lambdas, but IMapKeyRelation
is missing the index part.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or probably Element
will behave as you need.
@@ -1,26 +1,21 @@ | |||
<Project Sdk="Microsoft.NET.Sdk"> | |||
<Import Project="../../build-common/NHibernate.props" /> | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hum, probable Rider glitch... Bad point for that IDE.
|
||
public ValuesWrapper(PersistentGenericMap<TKey, TValue> map) | ||
{ | ||
_map = map; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just storing the persistent map, not the values, since the underlying map may still be changed at that point.
@@ -22,15 +25,22 @@ namespace NHibernate.Collection.Generic | |||
[DebuggerTypeProxy(typeof(DictionaryProxy<,>))] | |||
public partial class PersistentGenericMap<TKey, TValue> : AbstractPersistentCollection, IDictionary<TKey, TValue>, ICollection | |||
{ | |||
protected IDictionary<TKey, TValue> WrappedMap; | |||
protected IDictionary<TKey, TValue> WrappedMap { get; private set; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Protected only for one test file, looks like it should have been private but experienced a probable "rogue" change. At least switched to property with private setter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this please be reverted? And the test needs to change to use .Entries(...)
method instead.
public PersistentGenericMap() { } | ||
public PersistentGenericMap() | ||
{ | ||
_wrappedValues = new ValuesWrapper(this); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
x3, one for each constructor: they cannot call each over and the field cannot be initialized in place when using this
.
@@ -347,8 +358,7 @@ public ICollection<TValue> Values | |||
{ | |||
get | |||
{ | |||
Read(); | |||
return WrappedMap.Values; | |||
return _wrappedValues; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Read()
moved to ValuesWrapper
ICollection<TValue>
members. Otherwise, reaching Values
for calling AsQueryable
was loading the map.
I will not merge it myself since it would be a bit auto-approval of my own non cosmetic changes. |
36f5cb1
to
79f34ed
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reverted changes to test csproj.
Especially one rogue change causing all subsequent test to have NHibernate.SQL logger enabled.
28a4cb8
to
ac0275b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Just rebased.)
return GetEnumerator(); | ||
} | ||
|
||
public void Add(TValue item) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dictionary's "Values" collection is read-only.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Must still implement the interface, and I am just letting the throw be done by the underlying collection. But just for doing that, it triggers the read. Would you rather have a direct throw not reading at all? Since it is an error case anyway, I do not tend to optimize it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this actually matters much, so let it be this way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does not matter much but I have already changed it anyway, just not yet pushed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But how about "I do not tend to optimize it."?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requested changes done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dislike keeping a non private field naming on such a member, but well, if you prefer minimize changed lines count...
No description provided.