List of cleanProfiles() Examples

Examples of cleanProfiles()

de.anomic.search.Switchboard.cleanProfiles()
{@link CrawlProfiles Crawl Profiles} are saved independently from the queues themselvesand therefore have to be cleaned up from time to time. This method only performs the clean-up if - and only if - the {@link IndexingStack switchboard}, {@link LoaderDispatcher loader} and {@link plasmaCrawlNURL local crawl} queues are all empty.
Then it iterates through all existing {@link CrawlProfiles crawl profiles} and removesall profiles which are not hard-coded.

If this method encounters DB-failures, the profile DB will be reseted and true will be returned
@see #CRAWL_PROFILE_PROXY hardcoded @see #CRAWL_PROFILE_REMOTE hardcoded @see #CRAWL_PROFILE_SNIPPET_TEXT hardcoded @see #CRAWL_PROFILE_SNIPPET_MEDIA hardcoded @return whether this method has done something or not (i.e. because the queues have been filledor there are no profiles left to clean up) @throws InterruptedException if the current thread has been interrupted, i.e. by theshutdown procedure
net.yacy.search.Switchboard.cleanProfiles()
{@link CrawlProfiles Crawl Profiles} are saved independently from the queues themselvesand therefore have to be cleaned up from time to time. This method only performs the clean-up if - and only if - the {@link IndexingStack switchboard}, {@link LoaderDispatcher loader} and {@link plasmaCrawlNURL local crawl} queues are all empty.
Then it iterates through all existing {@link CrawlProfiles crawl profiles} and removesall profiles which are not hard-coded.

If this method encounters DB-failures, the profile DB will be reseted and true will be returned
@see #CRAWL_PROFILE_PROXY hardcoded @see #CRAWL_PROFILE_REMOTE hardcoded @see #CRAWL_PROFILE_SNIPPET_TEXT hardcoded @see #CRAWL_PROFILE_SNIPPET_MEDIA hardcoded @return whether this method has done something or not (i.e. because the queues have been filledor there are no profiles left to clean up) @throws InterruptedException if the current thread has been interrupted, i.e. by theshutdown procedure

Examples of de.anomic.search.Switchboard.cleanProfiles()

            showLimit = post.getInt("limit", 100);
            
            if (post.containsKey("clearcrawlqueue")) {
                final int c = sb.crawlQueues.noticeURL.stackSize(NoticedURL.StackType.LIMIT);
                sb.crawlQueues.noticeURL.clear(NoticedURL.StackType.LIMIT);
                try { sb.cleanProfiles(); } catch (final InterruptedException e) { /* Ignore this */}
                /*
                int c = 0;
                while (switchboard.urlPool.noticeURL.stackSize(plasmaCrawlNURL.StackType.LIMIT) > 0) {
                    urlHash = switchboard.urlPool.noticeURL.pop(plasmaCrawlNURL.StackType.LIMIT).hash();
                    if (urlHash != null) { switchboard.urlPool.noticeURL.remove(urlHash); c++; }

View Full Code Here

Examples of de.anomic.search.Switchboard.cleanProfiles()

                final String pattern = post.get("pattern", ".*").trim();
                final int option  = post.getInt("option", INVALID);
                if (".*".equals(pattern)) {
                    c = sb.crawlQueues.noticeURL.stackSize(NoticedURL.StackType.CORE);
                    sb.crawlQueues.noticeURL.clear(NoticedURL.StackType.CORE);
                    try { sb.cleanProfiles(); } catch (final InterruptedException e) {/* ignore this */}
                } else if (option > INVALID) {
                    try {
                        // compiling the regular expression
                        final Pattern compiledPattern = Pattern.compile(pattern);

View Full Code Here

Examples of de.anomic.search.Switchboard.cleanProfiles()

            showLimit = post.getInt("limit", 100);
            
            if (post.containsKey("clearcrawlqueue")) {
                final int c = sb.crawlQueues.noticeURL.stackSize(NoticedURL.StackType.REMOTE);
                sb.crawlQueues.noticeURL.clear(NoticedURL.StackType.REMOTE);
                try { sb.cleanProfiles(); } catch (final InterruptedException e) { /* Ignore this */}
                /*
                int c = 0;
                while (switchboard.urlPool.noticeURL.stackSize(plasmaCrawlNURL.StackType.LIMIT) > 0) {
                    urlHash = switchboard.urlPool.noticeURL.pop(plasmaCrawlNURL.StackType.LIMIT).hash();
                    if (urlHash != null) { switchboard.urlPool.noticeURL.remove(urlHash); c++; }

View Full Code Here

Examples of net.yacy.search.Switchboard.cleanProfiles()

            showLimit = post.getInt("limit", 100);
            
            if (post.containsKey("clearcrawlqueue")) {
                final int c = sb.crawlQueues.noticeURL.stackSize(NoticedURL.StackType.LIMIT);
                sb.crawlQueues.noticeURL.clear(NoticedURL.StackType.LIMIT);
                try { sb.cleanProfiles(); } catch (final InterruptedException e) { /* Ignore this */}
                /*
                int c = 0;
                while (switchboard.urlPool.noticeURL.stackSize(plasmaCrawlNURL.StackType.LIMIT) > 0) {
                    urlHash = switchboard.urlPool.noticeURL.pop(plasmaCrawlNURL.StackType.LIMIT).hash();
                    if (urlHash != null) { switchboard.urlPool.noticeURL.remove(urlHash); c++; }

View Full Code Here

Examples of net.yacy.search.Switchboard.cleanProfiles()

                final String pattern = post.get("pattern", ".*").trim();
                final int option  = post.getInt("option", INVALID);
                if (".*".equals(pattern)) {
                    c = sb.crawlQueues.noticeURL.stackSize(NoticedURL.StackType.CORE);
                    sb.crawlQueues.noticeURL.clear(NoticedURL.StackType.CORE);
                    try { sb.cleanProfiles(); } catch (final InterruptedException e) {/* ignore this */}
                } else if (option > INVALID) {
                    try {
                        // compiling the regular expression
                        final Pattern compiledPattern = Pattern.compile(pattern);

View Full Code Here

Examples of net.yacy.search.Switchboard.cleanProfiles()

            showLimit = post.getInt("limit", 100);
            
            if (post.containsKey("clearcrawlqueue")) {
                final int c = sb.crawlQueues.noticeURL.stackSize(NoticedURL.StackType.REMOTE);
                sb.crawlQueues.noticeURL.clear(NoticedURL.StackType.REMOTE);
                try { sb.cleanProfiles(); } catch (final InterruptedException e) { /* Ignore this */}
                /*
                int c = 0;
                while (switchboard.urlPool.noticeURL.stackSize(plasmaCrawlNURL.StackType.LIMIT) > 0) {
                    urlHash = switchboard.urlPool.noticeURL.pop(plasmaCrawlNURL.StackType.LIMIT).hash();
                    if (urlHash != null) { switchboard.urlPool.noticeURL.remove(urlHash); c++; }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.