Examples of org.locationtech.geogig.repository.WorkingTree$BlobInsertTask

Package org.locationtech.geogig.repository

Examples of org.locationtech.geogig.repository.WorkingTree$BlobInsertTask

org.locationtech.geogig.repository.WorkingTree
A working tree is the collection of Features for a single FeatureType in GeoServer that has a repository associated with it (and hence is subject of synchronization).
It represents the set of Features tracked by some kind of geospatial data repository (like the GeoServer Catalog). It is essentially a "tree" with various roots and only one level of nesting, since the FeatureTypes held in this working tree are the equivalents of files in a git working tree.
- A WorkingTree represents the current working copy of the versioned feature types
- A WorkingTree has a Repository
- A Repository holds commits and branches
- You perform work on the working tree (insert/delete/update features)
- Then you commit to the current Repository's branch
- You can checkout a different branch from the Repository and the working tree will be updated to reflect the state of that branch
@see Repository

            reader = new OsmosisReader(dataIn);
        } else {
            reader = new org.locationtech.geogig.osm.internal.XmlReader(dataIn, true, compression);
        }


        final WorkingTree workTree = workingTree();
        if (!add) {
            workTree.delete(OSMUtils.NODE_TYPE_NAME);
            workTree.delete(OSMUtils.WAY_TYPE_NAME);
        }


        final int queueCapacity = 100 * 1000;
        final int timeout = 1;
        final TimeUnit timeoutUnit = TimeUnit.SECONDS;
        // With this iterator and the osm parsing happening on a separate thread, we follow a
        // producer/consumer approach so that the osm parse thread produces featrures into the
        // iterator's queue, and WorkingTree.insert consumes them on this thread
        QueueIterator<Feature> iterator = new QueueIterator<Feature>(queueCapacity, timeout,
                timeoutUnit);


        ProgressListener progressListener = getProgressListener();
        ConvertAndImportSink sink = new ConvertAndImportSink(converter, iterator, platform(),
                mapping, noRaw, new SubProgressListener(progressListener, 100));
        reader.setSink(sink);


        Thread readerThread = new Thread(reader, "osm-import-reader-thread");
        readerThread.start();


        Function<Feature, String> parentTreePathResolver = new Function<Feature, String>() {
            @Override
            public String apply(Feature input) {
                if (input instanceof MappedFeature) {
                    return ((MappedFeature) input).getPath();
                }
                return input.getType().getName().getLocalPart();
            }
        };


        // used to set the task status name, but report no progress so it does not interfere
        // with the progress reported by the reader thread
        SubProgressListener noPorgressReportingListener = new SubProgressListener(progressListener,
                0) {
            @Override
            public void setProgress(float progress) {
                // no-op
            }
        };


        workTree.insert(parentTreePathResolver, iterator, noPorgressReportingListener, null, null);


        if (sink.getCount() == 0) {
            throw new EmptyOSMDownloadException();
        }

View Full Code Here

    }


    @Test
    public void testReportTreesEmptyTree() throws Exception {


        WorkingTree workingTree = geogig.getRepository().workingTree();
        workingTree.createTypeTree(linesName, linesType);


        List<DiffEntry> difflist = toList(diffOp.setReportTrees(true).setOldVersion(ObjectId.NULL)
                .setNewVersion(Ref.WORK_HEAD).call());


        assertNotNull(difflist);

View Full Code Here


        Feature lines1B = feature(linesType, idL2, "StringProp2_1", new Integer(1000),
                "LINESTRING (1 1, 2 2)");
        delete(lines1);
        // insert(lines2);
        WorkingTree workTree = repo.workingTree();
        Name name = lines1.getType().getName();
        String parentPath = name.getLocalPart();
        @SuppressWarnings("unused")
        Node ref = workTree.insert(parentPath, lines1B);
        geogig.command(AddOp.class).call();
        RevCommit commit2 = geogig.command(CommitOp.class).setAll(true).call();


        List<DiffEntry> diffs;
        diffOp.setOldVersion(commit1.getId());

View Full Code Here

        return Optional.fromNullable(report);


    }


    public OSMReport parseDiffFileAndInsert() {
        final WorkingTree workTree = workingTree();


        final int queueCapacity = 100 * 1000;
        final int timeout = 1;
        final TimeUnit timeoutUnit = TimeUnit.SECONDS;
        // With this iterator and the osm parsing happening on a separate thread, we follow a
        // producer/consumer approach so that the osm parse thread produces features into the
        // iterator's queue, and WorkingTree.insert consumes them on this thread
        QueueIterator<Feature> target = new QueueIterator<Feature>(queueCapacity, timeout,
                timeoutUnit);


        XmlChangeReader reader = new XmlChangeReader(file, true, resolveCompressionMethod(file));


        ProgressListener progressListener = getProgressListener();
        ConvertAndImportSink sink = new ConvertAndImportSink(target, context, workingTree(),
                platform(), new SubProgressListener(progressListener, 100));
        reader.setChangeSink(sink);


        Thread readerThread = new Thread(reader, "osm-diff-reader-thread");
        readerThread.start();


        // used to set the task status name, but report no progress so it does not interfere
        // with the progress reported by the reader thread
        SubProgressListener noProgressReportingListener = new SubProgressListener(progressListener,
                0) {
            @Override
            public void setProgress(float progress) {
                // no-op
            }
        };


        Function<Feature, String> parentTreePathResolver = new Function<Feature, String>() {
            @Override
            public String apply(Feature input) {
                return input.getType().getName().getLocalPart();
            }
        };


        workTree.insert(parentTreePathResolver, target, noProgressReportingListener, null, null);


        OSMReport report = new OSMReport(sink.getCount(), sink.getNodeCount(), sink.getWayCount(),
                sink.getUnprocessedCount(), sink.getLatestChangeset(), sink.getLatestTimestamp());
        return report;
    }

View Full Code Here

        }


    }


    private void applyPatch(Patch patch) {
        final WorkingTree workTree = workingTree();
        final StagingDatabase indexDb = stagingDatabase();
        if (reverse) {
            patch = patch.reversed();
        }


        List<FeatureInfo> removed = patch.getRemovedFeatures();
        for (FeatureInfo feature : removed) {
            workTree.delete(NodeRef.parentPath(feature.getPath()),
                    NodeRef.nodeFromPath(feature.getPath()));
        }
        List<FeatureInfo> added = patch.getAddedFeatures();
        for (FeatureInfo feature : added) {
            workTree.insert(NodeRef.parentPath(feature.getPath()), feature.getFeature());
        }
        List<FeatureDiff> diffs = patch.getModifiedFeatures();
        for (FeatureDiff diff : diffs) {
            String path = diff.getPath();
            DepthSearch depthSearch = new DepthSearch(indexDb);
            Optional<NodeRef> noderef = depthSearch.find(workTree.getTree(), path);
            RevFeatureType oldRevFeatureType = command(RevObjectParse.class)
                    .setObjectId(noderef.get().getMetadataId()).call(RevFeatureType.class).get();
            String refSpec = Ref.WORK_HEAD + ":" + path;
            RevFeature feature = command(RevObjectParse.class).setRefSpec(refSpec)
                    .call(RevFeature.class).get();


            RevFeatureType newRevFeatureType = getFeatureType(diff, feature, oldRevFeatureType);
            ImmutableList<Optional<Object>> values = feature.getValues();
            ImmutableList<PropertyDescriptor> oldDescriptors = oldRevFeatureType
                    .sortedDescriptors();
            ImmutableList<PropertyDescriptor> newDescriptors = newRevFeatureType
                    .sortedDescriptors();
            SimpleFeatureBuilder featureBuilder = new SimpleFeatureBuilder(
                    (SimpleFeatureType) newRevFeatureType.type());
            Map<Name, Optional<?>> attrs = Maps.newHashMap();
            for (int i = 0; i < oldDescriptors.size(); i++) {
                PropertyDescriptor descriptor = oldDescriptors.get(i);
                if (newDescriptors.contains(descriptor)) {
                    Optional<Object> value = values.get(i);
                    attrs.put(descriptor.getName(), value);
                }
            }
            Set<Entry<PropertyDescriptor, AttributeDiff>> featureDiffs = diff.getDiffs().entrySet();
            for (Iterator<Entry<PropertyDescriptor, AttributeDiff>> iterator = featureDiffs
                    .iterator(); iterator.hasNext();) {
                Entry<PropertyDescriptor, AttributeDiff> entry = iterator.next();
                if (!entry.getValue().getType().equals(TYPE.REMOVED)) {
                    Optional<?> oldValue = attrs.get(entry.getKey().getName());
                    attrs.put(entry.getKey().getName(), entry.getValue().applyOn(oldValue));
                }
            }
            Set<Entry<Name, Optional<?>>> entries = attrs.entrySet();
            for (Iterator<Entry<Name, Optional<?>>> iterator = entries.iterator(); iterator
                    .hasNext();) {
                Entry<Name, Optional<?>> entry = iterator.next();
                featureBuilder.set(entry.getKey(), entry.getValue().orNull());


            }


            SimpleFeature featureToInsert = featureBuilder.buildFeature(NodeRef.nodeFromPath(path));
            workTree.insert(NodeRef.parentPath(path), featureToInsert);


        }
        ImmutableList<FeatureTypeDiff> alteredTrees = patch.getAlteredTrees();
        for (FeatureTypeDiff diff : alteredTrees) {
            Optional<RevFeatureType> featureType;
            if (diff.getOldFeatureType().isNull()) {
                featureType = patch.getFeatureTypeFromId(diff.getNewFeatureType());
                workTree.createTypeTree(diff.getPath(), featureType.get().type());
            } else if (diff.getNewFeatureType().isNull()) {
                workTree.delete(diff.getPath());
            } else {
                featureType = patch.getFeatureTypeFromId(diff.getNewFeatureType());
                workTree.updateTypeTree(diff.getPath(), featureType.get().type());
            }
        }


    }

View Full Code Here

        assertTrue(feature.isPresent());
    }


    @Test
    public void testDeleteException() throws Exception {
        WorkingTree workTree = mock(WorkingTree.class);
        Context cmdl = mock(Context.class);
        when(cmdl.workingTree()).thenReturn(workTree);
        doThrow(new RuntimeException("Exception")).when(workTree).delete(any(String.class));
        ImportOp importOp = new ImportOp();
        importOp.setContext(cmdl);

View Full Code Here

        boolean overwrite = this.overwrite;
        if (alter) {
            overwrite = false;
        }


        final WorkingTree workTree = workingTree();


        RevFeatureType destPathFeatureType = null;
        final boolean destPathProvided = destPath != null;
        if (destPathProvided) {
            destPathFeatureType = this.command(ResolveFeatureType.class).setRefSpec(destPath)
                    .call().orNull();
            // we delete the previous tree to honor the overwrite setting, but then turn it
            // to false. Otherwise, each table imported will overwrite the previous ones and
            // only the last one will be imported.
            if (overwrite) {
                try {
                    workTree.delete(destPath);
                } catch (Exception e) {
                    throw new GeoToolsOpException(e, StatusCode.UNABLE_TO_INSERT);
                }
                overwrite = false;
            }
        }


        int tableCount = 0;


        for (String typeName : typeNames) {
            {
                tableCount++;
                String tableName = String.format("%-16s", typeName);
                if (typeName.length() > 16) {
                    tableName = tableName.substring(0, 13) + "...";
                }
                progressListener.setDescription("Importing " + tableName + " (" + tableCount + "/"
                        + typeNames.length + ")... ");
            }


            FeatureSource featureSource = getFeatureSource(typeName);
            SimpleFeatureType featureType = (SimpleFeatureType) featureSource.getSchema();


            final String fidPrefix = featureType.getTypeName() + ".";


            String path;
            if (destPath == null) {
                path = featureType.getTypeName();
            } else {
                NodeRef.checkValidPath(destPath);
                path = destPath;
                featureType = forceFeatureTypeName(featureType, path);
            }


            featureType = overrideGeometryName(featureType);


            featureSource = new ForceTypeAndFidFeatureSource<FeatureType, Feature>(featureSource,
                    featureType, fidPrefix);
            boolean hasPrimaryKey = hasPrimaryKey(typeName);
            boolean forbidSorting = !usePaging || !hasPrimaryKey;
            ((ForceTypeAndFidFeatureSource) featureSource).setForbidSorting(forbidSorting);


            if (destPathFeatureType != null && adaptToDefaultFeatureType && !alter) {
                featureSource = new FeatureTypeAdapterFeatureSource<FeatureType, Feature>(
                        featureSource, destPathFeatureType.type());
            }


            ProgressListener taskProgress = subProgress(100.f / typeNames.length);
            if (overwrite) {
                try {
                    workTree.delete(path);
                    workTree.createTypeTree(path, featureType);
                } catch (Exception e) {
                    throw new GeoToolsOpException(e, StatusCode.UNABLE_TO_INSERT);
                }
            }


            if (alter) {
                // first we modify the feature type and the existing features, if needed
                workTree.updateTypeTree(path, featureType);
                Iterator<Feature> transformedIterator = transformFeatures(featureType, path);
                try {
                    final Integer collectionSize = collectionSize(featureSource);
                    workTree.insert(path, transformedIterator, taskProgress, null, collectionSize);
                } catch (Exception e) {
                    throw new GeoToolsOpException(StatusCode.UNABLE_TO_INSERT);
                }
            }


            try {
                insert(workTree, path, featureSource, taskProgress);
            } catch (GeoToolsOpException e) {
                throw e;
            } catch (Exception e) {
                throw new GeoToolsOpException(e, StatusCode.UNABLE_TO_INSERT);
            }
        }


        progressListener.setProgress(100.f);
        progressListener.complete();
        return workTree.getTree();
    }

View Full Code Here


    /**
     * Inserts the feature to the index but does not stages it to be committed
     */
    protected ObjectId insert(GeoGIG geogig, Feature f) throws Exception {
        final WorkingTree workTree = geogig.getRepository().workingTree();
        Name name = f.getType().getName();
        String parentPath = name.getLocalPart();
        Node ref = workTree.insert(parentPath, f);
        ObjectId objectId = ref.getObjectId();
        return objectId;
    }

View Full Code Here


        return existed;
    }


    protected boolean delete(GeoGIG geogig, Feature f) throws Exception {
        final WorkingTree workTree = geogig.getRepository().workingTree();
        Name name = f.getType().getName();
        String localPart = name.getLocalPart();
        String id = f.getIdentifier().getID();
        boolean existed = workTree.delete(localPart, id);
        return existed;
    }

View Full Code Here

        } else {
            features = new EmptyFeatureReader<SimpleFeatureType, SimpleFeature>(getSchema());
        }


        String path = delegate.getTypeTreePath();
        WorkingTree wtree = getFeatureSource().getWorkingTree();


        GeoGigFeatureWriter writer;
        if ((flags | WRITER_ADD) == WRITER_ADD) {
            writer = GeoGigFeatureWriter.createAppendable(features, path, wtree);
        } else {

View Full Code Here

0 1 2 3 4 5

TOP

Related Classes of org.locationtech.geogig.repository.WorkingTree$BlobInsertTask

com.google.common.base.Stopwatch

com.google.common.util.concurrent.ThreadFactoryBuilder

com.vividsolutions.jts.geom.CoordinateSequenceFactory

com.vividsolutions.jts.geom.Envelope

com.vividsolutions.jts.geom.impl.PackedCoordinateSequenceFactory

org.geotools.data.Query

org.geotools.factory.Hints

org.geotools.feature.FeatureCollection

org.geotools.feature.FeatureIterator

org.locationtech.geogig.api.FeatureBuilder

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.