public final class NRTSuggester extends java.lang.Object implements Accountable
NRTSuggester executes Top N search on a weighted FST specified by a CompletionScorer
See lookup(CompletionScorer, Bits, TopSuggestDocsCollector)
for more implementation
details.
FST Format:
NOTE:
CompletionScorer.accept(int, Bits)
CompletionScorer.filtered
set to true
,
it is assumed that the filter will roughly filter out half the number of documents that match
the provided automatonModifier and Type | Class and Description |
---|---|
(package private) static class |
NRTSuggester.PayLoadProcessor
Helper to encode/decode payload (surface + PAYLOAD_SEP + docID) output
|
private static class |
NRTSuggester.ScoringPathComparator
Compares partial completion paths using
CompletionScorer.score(float, float) ,
breaks ties comparing path inputs |
Modifier and Type | Field and Description |
---|---|
private FST<PairOutputs.Pair<java.lang.Long,BytesRef>> |
fst
FST
NRTSuggesterBuilder.END_BYTE to denote the
end of the input
weight is a long
surface is the original, unanalyzed form followed by the docID |
private static long |
MAX_TOP_N_QUEUE_SIZE
Maximum queue depth for TopNSearcher
NOTE: value should be <= Integer.MAX_VALUE
|
private int |
maxAnalyzedPathsPerOutput
Highest number of analyzed paths we saw for any single
input surface form.
|
private int |
payloadSep
Separator used between surface form and its docID in the FST output
|
Modifier | Constructor and Description |
---|---|
private |
NRTSuggester(FST<PairOutputs.Pair<java.lang.Long,BytesRef>> fst,
int maxAnalyzedPathsPerOutput,
int payloadSep) |
Modifier and Type | Method and Description |
---|---|
private static double |
calculateLiveDocRatio(int numDocs,
int maxDocs) |
(package private) static long |
decode(long output) |
(package private) static long |
encode(long input) |
java.util.Collection<Accountable> |
getChildResources()
Returns nested resources of this class.
|
private static java.util.Comparator<PairOutputs.Pair<java.lang.Long,BytesRef>> |
getComparator() |
private int |
getMaxTopNSearcherQueueSize(int topN,
int numDocs,
double liveDocsRatio,
boolean filterEnabled)
Simple heuristics to try to avoid over-pruning potential suggestions by the
TopNSearcher.
|
static NRTSuggester |
load(IndexInput input,
CompletionPostingsFormat.FSTLoadMode fstLoadMode)
|
void |
lookup(CompletionScorer scorer,
Bits acceptDocs,
TopSuggestDocsCollector collector)
Collects at most
TopSuggestDocsCollector.getCountToCollect() completions that
match the provided CompletionScorer . |
long |
ramBytesUsed()
Return the memory usage of this object in bytes.
|
private static boolean |
shouldLoadFSTOffHeap(IndexInput input,
CompletionPostingsFormat.FSTLoadMode fstLoadMode) |
private final FST<PairOutputs.Pair<java.lang.Long,BytesRef>> fst
NRTSuggesterBuilder.END_BYTE
to denote the
end of the input
weight is a long
surface is the original, unanalyzed form followed by the docIDprivate final int maxAnalyzedPathsPerOutput
private final int payloadSep
private static final long MAX_TOP_N_QUEUE_SIZE
private NRTSuggester(FST<PairOutputs.Pair<java.lang.Long,BytesRef>> fst, int maxAnalyzedPathsPerOutput, int payloadSep)
public long ramBytesUsed()
Accountable
ramBytesUsed
in interface Accountable
public java.util.Collection<Accountable> getChildResources()
Accountable
getChildResources
in interface Accountable
Accountables
public void lookup(CompletionScorer scorer, Bits acceptDocs, TopSuggestDocsCollector collector) throws java.io.IOException
TopSuggestDocsCollector.getCountToCollect()
completions that
match the provided CompletionScorer
.
The CompletionScorer.automaton
is intersected with the fst
.
CompletionScorer.weight
is used to compute boosts and/or extract context
for each matched partial paths. A top N search is executed on fst
seeded with
the matched partial paths. Upon reaching a completed path, CompletionScorer.accept(int, Bits)
and CompletionScorer.score(float, float)
is used on the document id, index weight
and query boost to filter and score the entry, before being collected via
TopSuggestDocsCollector.collect(int, CharSequence, CharSequence, float)
java.io.IOException
private static java.util.Comparator<PairOutputs.Pair<java.lang.Long,BytesRef>> getComparator()
private int getMaxTopNSearcherQueueSize(int topN, int numDocs, double liveDocsRatio, boolean filterEnabled)
If a filter
is applied, the queue size is increased by
half the number of live documents.
The maximum queue size is MAX_TOP_N_QUEUE_SIZE
private static double calculateLiveDocRatio(int numDocs, int maxDocs)
private static boolean shouldLoadFSTOffHeap(IndexInput input, CompletionPostingsFormat.FSTLoadMode fstLoadMode)
public static NRTSuggester load(IndexInput input, CompletionPostingsFormat.FSTLoadMode fstLoadMode) throws java.io.IOException
java.io.IOException
static long encode(long input)
static long decode(long output)