Examples of org.apache.hadoop.mapred.gridmix.emulators.resourceusage.CumulativeCpuUsageEmulatorPlugin

Package org.apache.hadoop.mapred.gridmix.emulators.resourceusage

Examples of org.apache.hadoop.mapred.gridmix.emulators.resourceusage.CumulativeCpuUsageEmulatorPlugin

org.apache.hadoop.mapred.gridmix.emulators.resourceusage.CumulativeCpuUsageEmulatorPlugin

A {@link ResourceUsageEmulatorPlugin} that emulates the cumulative CPU usage by performing certain CPU intensive operations. Performing such CPU intensive operations essentially uses up some CPU. Every {@link ResourceUsageEmulatorPlugin} is configured with a feedback module i.e a {@link ResourceCalculatorPlugin}, to monitor the resource usage.

{@link CumulativeCpuUsageEmulatorPlugin} emulates the CPU usage in steps. The frequency of emulation can be configured via {@link #CPU_EMULATION_PROGRESS_INTERVAL}. CPU usage values are matched via emulation only on the interval boundaries.
{@link CumulativeCpuUsageEmulatorPlugin} is a wrapper program for managing the CPU usage emulation feature. It internally uses an emulation algorithm (called as core and described using {@link CpuUsageEmulatorCore}) for performing the actual emulation. Multiple calls to this core engine should use up some amount of CPU.

{@link CumulativeCpuUsageEmulatorPlugin} provides a calibration feature via {@link #initialize(Configuration,ResourceUsageMetrics,ResourceCalculatorPlugin,Progressive)} to calibrate the plugin and its core for the underlying hardware. As a result of calibration, every call to the emulation engine's core should roughly use up 1% of the total usage value to be emulated. This makes sure that the underlying hardware is profiled before use and that the plugin doesn't accidently overuse the CPU. With 1% as the unit emulation target value for the core engine, there will be roughly 100 calls to the engine resulting in roughly 100 calls to the feedback (resource usage monitor) module. Excessive usage of the feedback module is discouraged as it might result into excess CPU usage resulting into no real CPU emulation.

    // fake cpu usage generator
    FakeCpuUsageEmulatorCore fakeCore = new FakeCpuUsageEmulatorCore();
    fakeCore.setUnitUsage(unitCpuUsage);
    
    // a cumulative cpu usage emulator with fake core
    CumulativeCpuUsageEmulatorPlugin cpuPlugin = 
      new CumulativeCpuUsageEmulatorPlugin(fakeCore);
    
    // test with invalid or missing resource usage value
    ResourceUsageMetrics invalidUsage = createMetrics(0);
    cpuPlugin.initialize(conf, invalidUsage, null, null);
    
    // test if disabled cpu emulation plugin's emulate() call is a no-operation
    // this will test if the emulation plugin is disabled or not
    int numCallsPre = fakeCore.getNumCalls();
    long cpuUsagePre = fakeCore.getCpuUsage();
    cpuPlugin.emulate();
    int numCallsPost = fakeCore.getNumCalls();
    long cpuUsagePost = fakeCore.getCpuUsage();
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 numCallsPre, numCallsPost);
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 cpuUsagePre, cpuUsagePost);
    
    // test with valid resource usage value
    ResourceUsageMetrics metrics = createMetrics(targetCpuUsage);
    
    // fake monitor
    ResourceCalculatorPlugin monitor = new FakeResourceUsageMonitor(fakeCore);
    
    // test with default emulation interval
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test with custom value for emulation interval of 20%
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.2F);
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test if emulation interval boundary is respected (unit usage = 1)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(1);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.25F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    long initCpuUsage = monitor.getCumulativeCpuTime();
    long initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage, 
                          initNumCalls, "[no-op, 0 progress]");
    // test with 24% progress
    testEmulationBoundary(0.24F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[no-op, 24% progress]");
    // test with 25% progress
    //  target = 1000ms, target emulation at 25% = 250ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 4
    //  but current usage = init-usage = 100, hence expected = 100
    testEmulationBoundary(0.25F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[op, 25% progress]");
    
    // test with 80% progress
    //  target = 1000ms, target emulation at 80% = 800ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 410
    //  current-usage = init-usage = 100, hence expected-usage = 410
    testEmulationBoundary(0.80F, fakeCore, fakeProgress, cpuPlugin, 410, 410, 
                          "[op, 80% progress]");
    
    // now test if the final call with 100% progress ramps up the CPU usage
    testEmulationBoundary(1F, fakeCore, fakeProgress, cpuPlugin, targetCpuUsage,
                          targetCpuUsage, "[op, 100% progress]");
    
    // test if emulation interval boundary is respected (unit usage = 50)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(unitCpuUsage);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.40F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    initCpuUsage = monitor.getCumulativeCpuTime();
    initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage,

View Full Code Here

    // fake cpu usage generator
    FakeCpuUsageEmulatorCore fakeCore = new FakeCpuUsageEmulatorCore();
    fakeCore.setUnitUsage(unitCpuUsage);
    
    // a cumulative cpu usage emulator with fake core
    CumulativeCpuUsageEmulatorPlugin cpuPlugin = 
      new CumulativeCpuUsageEmulatorPlugin(fakeCore);
    
    // test with invalid or missing resource usage value
    ResourceUsageMetrics invalidUsage = createMetrics(0);
    cpuPlugin.initialize(conf, invalidUsage, null, null);
    
    // test if disabled cpu emulation plugin's emulate() call is a no-operation
    // this will test if the emulation plugin is disabled or not
    int numCallsPre = fakeCore.getNumCalls();
    long cpuUsagePre = fakeCore.getCpuUsage();
    cpuPlugin.emulate();
    int numCallsPost = fakeCore.getNumCalls();
    long cpuUsagePost = fakeCore.getCpuUsage();
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 numCallsPre, numCallsPost);
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 cpuUsagePre, cpuUsagePost);
    
    // test with valid resource usage value
    ResourceUsageMetrics metrics = createMetrics(targetCpuUsage);
    
    // fake monitor
    ResourceCalculatorPlugin monitor = new FakeResourceUsageMonitor(fakeCore);
    
    // test with default emulation interval
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test with custom value for emulation interval of 20%
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.2F);
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test if emulation interval boundary is respected (unit usage = 1)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(1);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.25F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    long initCpuUsage = monitor.getCumulativeCpuTime();
    long initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage, 
                          initNumCalls, "[no-op, 0 progress]");
    // test with 24% progress
    testEmulationBoundary(0.24F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[no-op, 24% progress]");
    // test with 25% progress
    //  target = 1000ms, target emulation at 25% = 250ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 4
    //  but current usage = init-usage = 100, hence expected = 100
    testEmulationBoundary(0.25F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[op, 25% progress]");
    
    // test with 80% progress
    //  target = 1000ms, target emulation at 80% = 800ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 410
    //  current-usage = init-usage = 100, hence expected-usage = 410
    testEmulationBoundary(0.80F, fakeCore, fakeProgress, cpuPlugin, 410, 410, 
                          "[op, 80% progress]");
    
    // now test if the final call with 100% progress ramps up the CPU usage
    testEmulationBoundary(1F, fakeCore, fakeProgress, cpuPlugin, targetCpuUsage,
                          targetCpuUsage, "[op, 100% progress]");
    
    // test if emulation interval boundary is respected (unit usage = 50)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(unitCpuUsage);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.40F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    initCpuUsage = monitor.getCumulativeCpuTime();
    initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage,

View Full Code Here

    // fake cpu usage generator
    FakeCpuUsageEmulatorCore fakeCore = new FakeCpuUsageEmulatorCore();
    fakeCore.setUnitUsage(unitCpuUsage);
    
    // a cumulative cpu usage emulator with fake core
    CumulativeCpuUsageEmulatorPlugin cpuPlugin = 
      new CumulativeCpuUsageEmulatorPlugin(fakeCore);
    
    // test with invalid or missing resource usage value
    ResourceUsageMetrics invalidUsage = createMetrics(0);
    cpuPlugin.initialize(conf, invalidUsage, null, null);
    
    // test if disabled cpu emulation plugin's emulate() call is a no-operation
    // this will test if the emulation plugin is disabled or not
    int numCallsPre = fakeCore.getNumCalls();
    long cpuUsagePre = fakeCore.getCpuUsage();
    cpuPlugin.emulate();
    int numCallsPost = fakeCore.getNumCalls();
    long cpuUsagePost = fakeCore.getCpuUsage();
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 numCallsPre, numCallsPost);
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 cpuUsagePre, cpuUsagePost);
    
    // test with valid resource usage value
    ResourceUsageMetrics metrics = createMetrics(targetCpuUsage);
    
    // fake monitor
    ResourceCalculatorPlugin monitor = new FakeResourceUsageMonitor(fakeCore);
    
    // test with default emulation interval
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test with custom value for emulation interval of 20%
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.2F);
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test if emulation interval boundary is respected (unit usage = 1)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(1);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.25F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    long initCpuUsage = monitor.getCumulativeCpuTime();
    long initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage, 
                          initNumCalls, "[no-op, 0 progress]");
    // test with 24% progress
    testEmulationBoundary(0.24F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[no-op, 24% progress]");
    // test with 25% progress
    //  target = 1000ms, target emulation at 25% = 250ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 4
    //  but current usage = init-usage = 100, hence expected = 100
    testEmulationBoundary(0.25F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[op, 25% progress]");
    
    // test with 80% progress
    //  target = 1000ms, target emulation at 80% = 800ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 410
    //  current-usage = init-usage = 100, hence expected-usage = 410
    testEmulationBoundary(0.80F, fakeCore, fakeProgress, cpuPlugin, 410, 410, 
                          "[op, 80% progress]");
    
    // now test if the final call with 100% progress ramps up the CPU usage
    testEmulationBoundary(1F, fakeCore, fakeProgress, cpuPlugin, targetCpuUsage,
                          targetCpuUsage, "[op, 100% progress]");
    
    // test if emulation interval boundary is respected (unit usage = 50)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(unitCpuUsage);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.40F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    initCpuUsage = monitor.getCumulativeCpuTime();
    initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage,

View Full Code Here

    // fake cpu usage generator
    FakeCpuUsageEmulatorCore fakeCore = new FakeCpuUsageEmulatorCore();
    fakeCore.setUnitUsage(unitCpuUsage);
    
    // a cumulative cpu usage emulator with fake core
    CumulativeCpuUsageEmulatorPlugin cpuPlugin = 
      new CumulativeCpuUsageEmulatorPlugin(fakeCore);
    
    // test with invalid or missing resource usage value
    ResourceUsageMetrics invalidUsage = createMetrics(0);
    cpuPlugin.initialize(conf, invalidUsage, null, null);
    
    // test if disabled cpu emulation plugin's emulate() call is a no-operation
    // this will test if the emulation plugin is disabled or not
    int numCallsPre = fakeCore.getNumCalls();
    long cpuUsagePre = fakeCore.getCpuUsage();
    cpuPlugin.emulate();
    int numCallsPost = fakeCore.getNumCalls();
    long cpuUsagePost = fakeCore.getCpuUsage();
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 numCallsPre, numCallsPost);
    
    //  test if no calls are made cpu usage emulator core
    assertEquals("Disabled cumulative CPU usage emulation plugin works!", 
                 cpuUsagePre, cpuUsagePost);
    
    // test with get progress
    float progress = cpuPlugin.getProgress();
    assertEquals("Invalid progress of disabled cumulative CPU usage emulation " 
                 + "plugin!", 1.0f, progress, 0f);
    
    // test with valid resource usage value
    ResourceUsageMetrics metrics = createMetrics(targetCpuUsage);
    
    // fake monitor
    ResourceCalculatorPlugin monitor = new FakeResourceUsageMonitor(fakeCore);
    
    // test with default emulation interval
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test with custom value for emulation interval of 20%
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.2F);
    testEmulationAccuracy(conf, fakeCore, monitor, metrics, cpuPlugin, 
                          targetCpuUsage, targetCpuUsage / unitCpuUsage);
    
    // test if emulation interval boundary is respected (unit usage = 1)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(1);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.25F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    long initCpuUsage = monitor.getCumulativeCpuTime();
    long initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage, 
                          initNumCalls, "[no-op, 0 progress]");
    // test with 24% progress
    testEmulationBoundary(0.24F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[no-op, 24% progress]");
    // test with 25% progress
    //  target = 1000ms, target emulation at 25% = 250ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 4
    //  but current usage = init-usage = 100, hence expected = 100
    testEmulationBoundary(0.25F, fakeCore, fakeProgress, cpuPlugin, 
                          initCpuUsage, initNumCalls, "[op, 25% progress]");
    
    // test with 80% progress
    //  target = 1000ms, target emulation at 80% = 800ms, 
    //  weighed target = 1000 * 0.25^4 (we are using progress^4 as the weight)
    //                 ~ 410
    //  current-usage = init-usage = 100, hence expected-usage = 410
    testEmulationBoundary(0.80F, fakeCore, fakeProgress, cpuPlugin, 410, 410, 
                          "[op, 80% progress]");
    
    // now test if the final call with 100% progress ramps up the CPU usage
    testEmulationBoundary(1F, fakeCore, fakeProgress, cpuPlugin, targetCpuUsage,
                          targetCpuUsage, "[op, 100% progress]");
    
    // test if emulation interval boundary is respected (unit usage = 50)
    //  test the case where the current progress is less than threshold
    fakeProgress = new FakeProgressive(); // initialize
    fakeCore.reset();
    fakeCore.setUnitUsage(unitCpuUsage);
    conf.setFloat(CumulativeCpuUsageEmulatorPlugin.CPU_EMULATION_PROGRESS_INTERVAL,
                  0.40F);
    cpuPlugin.initialize(conf, metrics, monitor, fakeProgress);
    // take a snapshot after the initialization
    initCpuUsage = monitor.getCumulativeCpuTime();
    initNumCalls = fakeCore.getNumCalls();
    // test with 0 progress
    testEmulationBoundary(0F, fakeCore, fakeProgress, cpuPlugin, initCpuUsage,

View Full Code Here

TOP

Related Classes of org.apache.hadoop.mapred.gridmix.emulators.resourceusage.CumulativeCpuUsageEmulatorPlugin

org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulators

java.util.Random

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.